Europe PMC

This website requires cookies, and the limited processing of your personal data in order to function. By using the site you are agreeing to this as outlined in our privacy notice and cookie policy.

Measures of Mentoring, Department Climate, and Graduate Student Preparedness in the Responsible Conduct of Psychological Research.

Author information

Affiliations

  • 1. Fordham University Center for Ethics Education.
      Authors
    • Fisher CB 1
    (1 author)

ORCIDs linked to this article

Ethics & Behavior, 01 May 2009, 19(3):227-252
https://doi.org/10.1080/10508420902886726 PMID: 23626454 PMCID: PMC3634717

Free full text in Europe PMC

Abstract 

Drawing upon two independent national samples of 201 and 241 psychology graduate students, this article describes the development and psychometric evaluation of 4 web-based student self-report scales tapping student socialization in the responsible conduct of research (RCR) with human participants. The Mentoring the Responsible Conduct of Research Scale (MRCR) is composed of 2 subscales assessing RCR instruction and modeling by research mentors. The 2 subscales of the RCR Department Climate Scale (RCR-DC) assess RCR department policies and faculty and student RCR practices. The RCR-Preparedness Scale (RCR-P) and the RCR Field Integrity scale (RCR-FI) measure respectively students' confidence in their ability to conduct research responsibly and their belief in the RCR integrity of psychology as a discipline. Factor analysis, coefficient alphas, correlations and multiple regression analyses demonstrated each of the scales had good internal consistency and concurrent and construct validity.

Free full text 

Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Ethics Behav. Author manuscript; available in PMC 2013 Apr 24.
Published in final edited form as:
Ethics Behav. 2009 May 1; 19(3): 227–252.
Published online 2009 May 13. https://doi.org/10.1080/10508420902886726
PMCID: PMC3634717
NIHMSID: NIHMS298914
PMID: 23626454

Measures of Mentoring, Department Climate, and Graduate Student Preparedness in the Responsible Conduct of Psychological Research

Celia B. Fisher, Adam L. Fried, Sabrina J. Goodman, and Kaori Kubo Germano
Author information Copyright and License information Disclaimer

Abstract

Drawing upon two independent national samples of 201 and 241 psychology graduate students, this article describes the development and psychometric evaluation of 4 web-based student self-report scales tapping student socialization in the responsible conduct of research (RCR) with human participants. The Mentoring the Responsible Conduct of Research Scale (MRCR) is composed of 2 subscales assessing RCR instruction and modeling by research mentors. The 2 subscales of the RCR Department Climate Scale (RCR-DC) assess RCR department policies and faculty and student RCR practices. The RCR-Preparedness Scale (RCR-P) and the RCR Field Integrity scale (RCR-FI) measure respectively students’ confidence in their ability to conduct research responsibly and their belief in the RCR integrity of psychology as a discipline. Factor analysis, coefficient alphas, correlations and multiple regression analyses demonstrated each of the scales had good internal consistency and concurrent and construct validity.

Keywords: Ethics, Research, Responsible Conduct of Research, Mentoring, Department Climate, Graduate Education

Across scientific disciplines, there is a consensus that graduate training in the responsible conduct of research (RCR) is vital to the continued growth of the field (Kalichman, 2007; NAS, 1997; 2002; Steneck, 2007). In 2005, the number of research doctorates awarded in the United States reached an all-time high, with 43,354 recipients earning their doctorates from 416 colleges and universities (Hoffer et al., 2006). Within the social science fields, psychology accounted for almost half of doctorates awarded in 2005, and was second only to biology in the percentage (7.7%) of doctorates awarded across all disciplines. Relevant to this increase in the number of research scientists is the concurrent and steady rise of federal regulations and field specific codes of conduct established to ensure that research is conducted responsibly (45CFR46; DHHS, 2005). In response to growing public awareness and federal oversight of the roles and responsibilities of scientists, interest in the ways in which RCR practices are communicated to students in graduate programs has increased.

Psychology has long been self-reflective about the responsible conduct of research (e.g., APA, 1973; Baumrind, 1964; Fisher & Tryon, 1990; Haney, Banks & Zimbardo, 1973; Keith-Spiegel & Koocher, 1985; Sales & Folkman, 2000; Seiber, 1982; Smith, 1976). The first ethics code issued by the American Psychological Association (APA) included research ethics as one of its major sections (APA, 1953). In the half century since its initial publication revisions of the APA Ethics Code have evolved to include specific standards corresponding to the ethical challenges encountered by psychologists and written into federal regulations for research protections. The most recent version of the Ethics Code includes standards comparable to the core RCR elements recommended by the National Academy of Sciences and the Office of Research Integrity (NAS, 1997; ORI, 2000) including compliance with institutional review boards (IRBs), informed consent, integrity in data collection and analysis, avoidance of plagiarism, honesty in dissemination of research results, protection of human participants and animal welfare, debriefing, collaboration with colleagues, and avoidance of conflicts of interest (APA, 2002; Fisher, 2003).

Graduate Education in the Responsible Conduct of Research

There is increasing pressure on academic and research institutions to ensure adequate RCR training among students and research staff. This trend is due in part to increased public awareness of the impact of research on public health policies and practices (Eisen & Berry, 2002;) and the requirement by the Department of Health and Human Services (DHHS) that principal investigators and research-supported staff acquire RCR training (PHS, 2000; 2001). Although federal regulations and APA ethical standards for research have become increasingly explicit in prescriptions and prohibitions, RCR socialization in psychology has remained largely implicit, reflecting the apprenticeship model characterizing graduate training in the sciences in general (Swazey & Anderson, 1996). For example, ethics instruction has not been a formal aspect of graduate education in psychological science programs. Moreover, while a course covering ethics is required in all practitioner-based APA accredited programs, the specific coverage of research ethics is not standardized (Fisher, Wertz & Goodman, 2009). As a consequence, there is a paucity of empirical knowledge on the formal and informal mechanisms used by mentors and psychology departments to socialize graduate students in the field's research ethics values and procedures.

The goal of the research described in this article was to develop and validate a set of scales that can contribute to our understanding of psychology graduate student RCR socialization. In the next section, we briefly review literature providing the conceptual basis for our scale development.

RCR Mentoring, Department Climate and Students’ Preparedness and Attitudes

Psychology graduate programs are the primary training ground for student socialization in the responsible conduct of research. At the individual level, research mentors, through their instruction and actions, expose graduate students to the ways in which research is ethically practiced in the field. At the organizational level, each graduate department, through its policies and guidelines, creates its own RCR climate in which graduate students are socialized in the ethical conduct of research (CRI, 2003; NAS, 2002; PHS, 2000; Steneck, 2001).

Mentoring

While modern definitions of the role of a mentor in academic research settings have varied across and within scientific disciplines (Clark, Harden & Johnson, 2000; Council of Graduate Schools, 1995; NAS, 1992; 1997; Swazey & Anderson, 1996), they typically include the role of the mentor in transmitting knowledge and overseeing the mentee's professional and personal development (Johnson & Nelson, 1999). Motta (2002) proposed the mentor-mentee relationship was the “life force” (p. w1) of the academic research enterprise, and argued that it is through the strengthening of this relationship that the promotion of RCR occurs. In science, awareness and understanding of professional standards, regulations, and ethical values of the community are integral to success, and mentors transmit this information to their protégés explicitly and implicitly through the mentoring process (Bird, 2001; Magnus & Kalichman, 2007).

Explicit Transmission of RCR Values & Practices

Explicit transmission of knowledge occurs when the mentor provides direct instructions or practical guidance to steer the protégé toward good RCR practice. Effective psychology mentors have often been described as available and invested, altruistic, ethical and intentional role models (Gilbert, 1985; Kitchener, 1992; Wilson & Johnson, 2001). Additionally, mentors provide protégés with protection when necessary (Kitchener, 1992), helping them to avoid, or overcome mistakes. Mentors offer the practical guidance that is usually not available in the classroom, training the protégé in the application of RCR policies and its underlying philosophies during the course of the research endeavor.

Implicit Transmission of RCR Values & Practices

While the explicit teaching of RCR policy and its underlying philosophy is an integral component of mentoring, it alone cannot teach young researchers what behaviors constitute good RCR. It is often the case that the default method for teaching good research practices is through “unwitting and serendipitous example” (Magnus & Kalichman, 2007, p. 1). Mentors have the added responsibility of modeling the exemplary RCR behaviors they seek to teach (Swazey & Anderson, 1996) and offering first-hand examples of academic and professional competence and excellence. It is through this “hidden curriculum” (Hafferty & Franks, 1994, p. 770) that students implicitly learn “how it really works” (Fryer-Edwards, 2002, p.58). However, as Stern and Elliot (1997) have argued, while informal modeling provides students with guidance on what to do, it does not teach them why they are doing it. Without understanding the underlying reasoning of RCR practices, psychology students may be ill-equipped to generalize RCR decision-making to new research contexts (Eisen & Berry, 2002). Thus, to be effective, mentoring must encompass both the explicit and implicit transmission of RCR knowledge.

As important as mentoring is to ensuring the transmission of principles, values, policies and standards of scientific integrity, it has rarely been a focus of empirical examination across scientific disciplines for several reasons (Bird, 2001). First, studying RCR practice in general and mentoring in particular may be complicated by the fact that research procedures that are implemented specifically to address ethical issues are under-reported in research publications. Second, evidence of poor ethics training is difficult to operationalize due to the infrequency of reports of psychological scientific misconduct reaching levels of public awareness. Finally, mentoring functions related to research training in general and RCR in particular differ across distinct scientific disciplines (Brown & Kalichman, 1998; Swazey & Anderson, 1996; Wright & Komparens, 1998), making it difficult to operationalize a universal set of general standards of research training and RCR practices.

Department Research Ethics Climate

An equally important component of RCR socialization is the graduate program department climate, which can be defined as the shared values and attitudes of organizational members (Ashforth, 1985). Victor and Cullen (1988) called organizations “social actors responsible for the ethical or unethical behaviors of their employees” (p. 101), and this is true in academic institutions, as the climate fostered within departments provides rich information about the ethical behaviors, moral obligations, and codes or regulations to which students and all members of the profession are expected to adhere. Studies have provided evidence for the significant influence of department climate on the attitudes of institutional research department members (Ashforth, 1985), and in turn, on the socialization of professional values in graduate students (Anderson & Louis, 1994). The National Research Council (NAS, 2002) emphasized that because the research mentor is an unpredictable variable in RCR training, the institutional climate is charged with being a constant resource, providing consistent and effective training and education, policies and procedures, and tools and support systems. The importance of the role the department climate plays on RCR practices was underscored by Anderson, Louis, and Earle (1994), who found it to be the strongest predictor for misconduct in graduate students across the science disciplines.

Explicit RCR Department Policies

Similar to the mentoring process, there are explicit and implicit components of the department climate. The explicit components are formal department RCR policies and resources. These include: requiring a research ethics course as a component of the graduate curriculum, submission of student research for IRB review and efforts to ensure student and faculty awareness of policies for reporting suspected ethical violations, and that ethics complaints are adjudicated responsibly and fairly. In one of the few studies examining the effects of a research ethics curriculum on graduate students in psychology, Tryon (2002) found that school psychology doctoral students who had taken an ethics course felt significantly more prepared to obtain informed consent and assent from parent, adolescent and child participants for their dissertation and/or other research than students who had not had a course.

Implicit RCR Department Practices

The implicit components of the RCR department climate are transmitted through department-wide faculty and student behaviors that indicate respect and compliance with the research ethics values of the discipline (NAS, 2002). In describing the research training environment for counseling psychologists, Gelso (2006) noted that faculty members have the most effective power over the environment, and thus, have the most responsibility for promoting good RCR practices. Large scale surveys of science faculty across disciplines have found that while the majority of faculty and students surveyed agreed that faculty should have a collective responsibility for the professional-ethical conduct of their graduate students, a majority also recognized a gulf between what should be and what is actually exercised (Louis, Anderson, & Rosenberg, 1995; Swazey, Anderson & Louis, 1993).

Student RCR Preparedness and Attitudes toward the Discipline

Ideally, an outcome of positive RCR mentoring and department climate are students who feel prepared to engage in responsible research practices as they progress from the protective environment of their training institution to work in settings that expect them to independently establish research programs. There is a paucity of empirical data on students’ sense of preparedness to conduct research in general and research ethics in particular (exceptions include Brown & Kalichman, 1998, and Tryon, 2002). The extent to which students feel prepared upon graduation to independently submit IRB protocols, construct adequate informed consent and confidentiality procedures, debrief participants, collaborate with other scientists, avoid conflicts of interest and report research results honestly is likely to influence whether they pursue careers involving research and if they do, whether they are able to conduct research responsibly.

Through both the “hidden curriculum” (Fryer-Edwards, 2002, p. 58). provided by their research mentors and the RCR polices and practices of their graduate departments, students construct beliefs about the values and integrity of their discipline. Confidence in the RCR integrity of the profession is motivation for making post-graduate career choices that include research activities and for conducting such activities responsibly. By contrast, perceiving psychological science as lacking an ethical core can discourage psychology graduates from pursuing research careers and encourage disregard for ethical principles in their own research endeavors.

The Current Study

There is little empirical data on RCR socialization in the sciences in general and in psychology specifically. The purpose of this study was to construct and psychometrically evaluate scales that could assist in generating an empirically based understanding of components of RCR mentoring and department climate that contribute to psychology graduate students’ sense of preparedness to conduct psychological science with human participants responsibly and their confidence in the research integrity of psychology as a discipline.

Scale development and validation was conducted in two phases and included expert ratings for item construction and content validity, as well as item analysis, factor analysis, inter-item reliability, correlation and multiple regression analysis. Drawing on the small but growing body of literature on RCR in general and psychological science in particular to test for concurrent and construct validity, we hypothesized the following:

  • Items on scales constructed to measure RCR mentoring and department climate would cluster in multidimensional components reflecting implicit and explicit communication of research ethics values and practices.

  • RCR mentoring, department climate, and preparedness scales would be correlated with independent items of mentor and department satisfaction and with existing scales measuring research mentoring, department research climate, and research efficacy, respectively.

  • There would be positive associations between the RCR scales and independent questions tapping satisfaction with research mentoring, the research ethics department climate, and students’ psychology program in general.

  • Students whose mentors were selected rather than assigned by the department, those with more years with their mentor, and with more publications would be more likely to rate their mentor higher in both explicit and implicit RCR mentoring behaviors, but these factors would not contribute to scores tapping RCR department climate.

  • Students whose programs required an ethics course would have higher scores on scales tapping RCR department policy, RCR preparedness, and confidence in the RCR integrity of the field.

  • Students with a higher number of authored publications, who had or were closer to receiving their doctorate and who had taken an ethics course would have higher scores on RCR preparedness, but these variables would not be associated with students’ confidence in the RCR integrity of the field of psychology.

  • RCR mentoring and department climate RCR subscales would be positively correlated with scales constructed to tap students’ sense of the preparedness to conduct research responsibly and their confidence in the RCR integrity of the discipline of psychology.

General Method

Participants

National samples of 201 graduate and postgraduate students (71% female; 81% non-Hispanic white; Mean age = 30 years, SD = 5.8) in Phase 1 and 241 students (77% female; 83% non-Hispanic white; Mean age = 29 years, SD = 4.0) in Phase 2, from geographically and mission-diverse graduate programs were recruited for participation in this study. To be included, students had to be in or have recently graduated from an MA or Ph.D. psychology program and to have conducted a graduate research project including human participants. On average, current students in Phases 1 and 2 had spent a total of 3.6 years (SD = 2.9) and 4.57 years (SD = 2.65), respectively, in their program and 2.5 (SD = 2.1) and 3 (SD = 1.7) years with their mentor. The over-representation of females and non-Hispanic white students parallel those reported in the field (Hoffer et al., 2006). Since nationally a greater proportion of psychology students are in scientist/practitioner or practitioner programs, efforts were made to over-sample students in basic or applied research programs through outreach to department chairs and program directors. Detailed data on student, mentor, and department characteristics are presented in Table 1.

Table 1

Student, Mentor, and Program Characteristics in Phases 1 and 2

Phase 1Phase 2
N = 201N = 241
n % n %
Student Characteristics
Gender
    Female14471.6%18576.8%
    Male5728.4%5623.2%
Ethnicity
    American Indian/Alaska Native31.5%10.4%
    Black63.0%125.0%
    East Asian/Southeast Asian168.0%166.6%
    Hispanic/Latino199.6%93.7%
    Native Hawaiian/Pacific Islander52.5%10.4%
    Non-Hispanic White16381.9%20083.0%
    Other73.5%20.8%
Degree Program
    Ph.D.18391.0%21187.6%
    Psy.D.e6.0%2711.2%
    Ed.D.31.5%00.0%
    Other31.5%31.2%
Student Statusa
    MA in process4522.3%5422.4%
    MA completed12662.7%13756.8%
    Ph.D. in process9145.3%11547.7%
    Ph.D. completed5326.4%3815.8%
Primary Career Goal
    Research----8836.5%
    Teaching----3414.1%
    Professional Practice----11246.5%
    Industrial/Organizational----72.9%
Student Publications
    None6934.3%8535.3%
    1-26531.3%8334.5%
    3 or more6733.4%7330.2%
Has taken a class that included Research Ethics----12953.0%
Department Characteristics
Geographic Region
    West (Pacific & Mountain states)157.5%197.2%
    Midwest (Midwest/Central states)63.0%5320.0%
    Northeast2311.4%10338.9%
    Southeast10.5%9034.0%
    Unidentified15677.6%00.0%
Research Ethics Course
    Required8341.3%14660.6%
    Elective5728.4%2811.6%
    Not Offered6130.3%6727.8%
Program
    Clinical Psychology5226.0%11045.6%
    Counseling Psychology31.5%187.5%
    Cognitive Psychology2110.4%83.3%
    Community Psychology00.0%83.3%
    Developmental Psychology2512.0%2811.6%
    Experimental Psychology3617.9%93.7%
    Forensic Psychology105.0%00.0%
    Industrial/Organizational Psychology52.5%62.5%
    Neuropsychology/Neuroscience73.5%20.8%
    School Psychology199.5%187.5%
    Social and Personality Psychology136.5%229.0%
    Other105.0%125.0%
Number of Faculty in Graduate Program
    ≤ 2011557.3%19179.2%
    > 208642.7%5020.8%
Mentor Characteristics
Gender
    Female7034.8%9840.7%
    Male13165.2%14359.3%
Ethnicity
    American Indian/Alaska Native84.0%41.7%
    Black84.0%31.2%
    East Asian/Southeast Asian2211.0%125.0%
    Hispanic/Latino94.5%72.9%
    Native Hawaiian/Pacific Islander52.5%20.8%
    Non-Hispanic White16381.5%21890.5%
    Other/Unknown10.5%72.9%
Professional Status
    Junior Level02713.4%4317.8%
    Mid Level15326.4%6225.7%
    Senior Level211858.7%12953.5%
Mentor
    Selected16682.6%20987.0%
    Assigned3517.4%3213.0%
aNot all program required a master's project; there is an overlap between MA completion and doctoral progress.
0The example provided to participants was: “e.g., assistant professor, research associate”
1The example provided to participants was, “e.g., associate professor, senior researcher”
2The example provided to participants was, “e.g., full professor, director of research”

Item Development

Four distinct Internet-based scales were developed to measure RCR mentoring, department climate, student preparedness, and student confidence in the RCR integrity of the discipline. Items and format were developed based upon theoretical literature, federal regulations, NAS and ORI guidelines, standards in the APA Ethics Code, and empirically tested scales tapping mentoring, department climate, and preparedness in research design and implementation across different science disciplines.

Content validity

In Phase 1, items for the initial versions of the scales tapping RCR mentoring, department climate, and student preparedness were evaluated by 7 faculty experts in research ethics from 5 different universities and by 7 psychology doctoral students from Fordham University's graduate programs in Psychometrics, Developmental, and Clinical Psychology. All student raters had taken the Department's required course in Ethics in Psychology and had been mentored in a project involving human subject data collection. For each instrument, experts and students rated each item on a scale of 1= not appropriate, 2 = appropriate, 3 = highly appropriate. For items rated ‘1,’ student and faculty experts were asked to recommend either deletion of the item or rewording that would earn the item a ‘2’ or ‘3’ rating. Faculty experts were also asked to write items for any essential RCR categories they believed were missing from the instruments. Items recommended for deletion and not reworded by more than 1 expert were eliminated, and suggested items were added. Raters were also asked to comment on scale organization, format, and item clarity.

RCR Scales

In this section, we describe the format and general content of the 4 RCR scales constructed for and evaluated in Phases 1 and 2. The items maintained in the final versions of the scales are provided in the Results section. The wording of each item was carefully constructed so that it appropriately reflected the wording of the relevant standard. For example, in the Responsible Conduct of Research Department Climate Index the item, “there is a written policy on how to avoid conflicts of interest in research,” reflects the language used in the Ethics Code whereby psychologists are mandated to “refrain from” engaging in relationships that may lead to conflicts of interest (Standard 3.06; APA, 2002).

Mentoring the Responsible Conduct of Research Scale (MRCR)

The format and content of the MRCR drew upon scales focused on research mentoring for general sciences (Swazey & Anderson, 1996) and clinical and counseling psychology (Clark et al., 2000; Hollingsworth & Fassinger, 2002). The Phase I version of the MRCR consisted of 26 items for which students responded on a 6-point Likert-type scale (1 = Strongly Disagree; 6 = Strongly Agree) to statements describing the RCR related behaviors of their mentor. Mentor was defined as the “faculty member who has/had the primary responsibility for supervising your master's, doctoral, or other graduate level independent psychology research.” Students were also instructed that if they had more than 1 research mentor they should “select the mentor you believe had the greatest influence (positive or negative) on your development as a researcher.” The MRCR scale revised for Phase 2 consisted of 2 subscales. The MRCR-Instruction (MRCR-I) subscale asked students to respond using a 6-point Likert-type scale (1 = Extremely Unhelpful, 6 = Extremely Helpful) to 23 statements completing the phrase “My research mentor gave me helpful training about...” The statements described specific RCR procedural content, e.g. “Appropriate informed consent procedures”, and “How to protect participant confidentiality.” The MRCR-Modeling (MRCR-M) subscale asked students to respond using a 6-point Likert-type scale (1 = Extremely False, 6 = Extremely True) to 23 statements completing the phrase “My research mentor....” These subscale items described the extent to which the mentor engaged in RCR behaviors and encouraged the mentee to do so, e.g. “Conducted his/her own research ethically,” and “Encouraged me to consider ethical issues relevant to my research.”

The Responsible Conduct of Research Department Climate Index (RCR-DC)

The format of the RCR-DC was adapted from questions developed by The Acadia Institute and applied by Michigan State University to assess the ethical environment for science program research and graduate studies (Swazey, Anderson & Louis, 1993; Wright & Klomparens, 1998) and in Victor and Cullen's (1988) study of ethical work climates. Using a 6-point Likert-type scale (1 = Strongly Disagree, 6 = Strongly Agree) in Phase 1, students responded to 18 items following the statement “In my program.” For Phase 2 the stem was modified to “In my graduate psychology department” and items increased to 38 to adequately represent department RCR policies (e.g. “There is a clear policy for handling research ethics complaints”) and department faculty and student RCR behaviors (e.g., “Faculty and students engage in ethically questionable research practices”).

The RCR Perceived Preparedness Scale (RCR-P)

The 23-item RCR-P drew from the APA Ethics Code, items from Tryon's (2002) of school psychology students’ preparation o address a wide range of ethical issues and Brown and Kalichman's (1998) RCR competence questionnaire developed for graduate students across 11 scientific disciplines, including psychology. No items were added or deleted following analyses of Phase 1 data, although a few were reworded to enhance clarity. In response to the stem, “At this point in my research career I feel my graduate training has prepared me to...,” students rated on a 6-point Likert-type response format (1 = Extremely False, 6 = Extremely True) items such as “Maintain research records in a manner consistent with APA ethical standards,” and “Assign appropriate authorship credit for publications.”

The RCR Perceived Field Integrity Scale (RCR-FI)

For Phase 2 we constructed a scale to measure graduate students’ confidence in the research integrity of the field of psychology. We found no related scales for this purpose and thus based the content of the items on the APA Ethics Code Introduction, Principles, and Standards. Students responded to 13 items on a 6-point Likert-type scale (1 = Extremely false, 6 = Extremely True) following the statement “Based on my psychology graduate research training, I believe...” Examples of items are: “It is common practice for research psychologists to consider and resolve ethical problems,” “Psychologists tend to disregard ethical dilemmas that arise in their research.”

Measures Included for Scale Validation

In Phase I three existing measures of research mentoring, department climate and student efficacy that did not include any RCR items were administered to establish construct validity of the RCR scales. We hypothesized that scores on these scales, indicating higher levels of research mentoring (AWAI-S), department climate (RTES-R-S), and efficacy (SERM), would be positively related to scores on the MRCR, RCR-DC, and RCR-P, respectively.

The Advisory Working Alliance Index- Student Version

(AWAI-S; Schlosser & Gelso, 2001) is a 48-item self-report measure designed to assess graduate students’ perceptions of the working alliance with their advisor along three dimensions: rapport, apprenticeship, and identification-individuation. For the purposes of this study, “Mentor” was substituted for “Advisor” and items were re-worded into the past tense. The AWAI-S Rapport subscale was included in Phase 2 to distinguish modified MRCR subscales.

The Research Training Environment Scale—Revised-Short Form

(RTES-R-S; Kahn & Miller, 2000), The RTES-R-S is an 18-item scale assessing interpersonal and instructional dimensions of the graduate training environment in clinical counseling and school psychology graduate programs (Gelso, Mallinckrodt & Judge, 1996; Gelso et al., 1983; Kahn & Gelso, 1997; Kahn & Scott, 1997; Mallinckrodt et al., 1990; Royalty et al., 1986). Each item is rated on a 5-point Likert-type scale ranging from 1 = Disagree to 5 = Agree.

The Self-Efficacy in Research Measure

(SERM; Kahn & Scott, 1997) is a 12-item scale assessing psychology graduate students’ perceptions of their competencies in research design, practical, writing, and quantitative skills. Participants are asked to rate their degree of confidence on a scale from 0 to 9 (0 = No Confidence, 9 = Total Confidence) in their ability to carry out research-related tasks such as ”Formulating hypotheses” and “Using statistical packages (e.g., SPSS-X, SAS, etc).

Demographic Information

Demographic information regarding students’ gender, ethnicity, age, year in graduate school, research experience, type of graduate program, mentor and department characteristics, and overall satisfaction with their mentor, RCR department climate, and graduate program in general was collected to both describe the participant population and determine if scores on the scales were associated with specific demographic characteristics in predicted directions.

Procedures

Participants were recruited through e-mail blasts, posters, and flyers distributed to students by faculty at graduate programs across the United States. These announcements directed students to a web site describing the study and provided a username and password that students needed to access the anonymous online survey. Over time students from other parts of the country became aware of the study and logged on to the web site to participate. To protect anonymity, the web site was constructed with firewalls to prevent anyone (including the investigators) from identifying participants’ Internet Protocol (IP) addresses. The study was approved by the University Institutional Review Board (IRB) and students viewed the informed consent information on the home page of the web site prior to beginning the survey. Since the survey was anonymous and students could withdraw at any time prior to submitting the completed survey, submission of the completed survey was considered to reflect student consent. Immediately after students submitted the survey they were directed to an email address to request their $30 gift card, which was emailed to them. The email address could not be traced to the student's survey response.

Results

For all scales negative items were reverse-scored for analyses. Item analyses followed procedures suggested by Fisherman and Galguera (2003). The goal of item analysis was to determine in Phase 1 which items should be retained, deleted or modified and in Phase 2 which items should be maintained. To assess the degree to which each item contributed to reliability of the measure as a whole, we used the statistic alpha-if-item-deleted. Items that would raise the scale alpha level if deleted were considered for exclusion or revision. Item difficulty scores were generated to identify items with overly skewed distributions representing floor or ceiling effects. Optimal items were those that yielded a mean score near the mathematical midpoint of each scale and approximated normal distributions around the mean score for that item. Item discriminability was evaluated utilizing SPSS's “corrected item total correlation” which is derived from the subtraction of the item response from the total score to which it has contributed.

The literature suggested that RCR mentoring and department climate might be multidimensional in their influence on student socialization. In Phase 1 we conducted separate exploratory factor analyses using varimax rotation for the MRCR and RCR-DC. Based on perusal of the initial Scree plots, 2 factors were extracted for each instrument. A minimum component loading of .30 was required for items to be selected and items meeting the .30 criteria must also have been at least .15 greater than all other items loading on the same component. Scale revisions were made based on these analyses and tested in Phase 2. In Phase 2, additional factor analyses were performed using a factor loading minimum criteria of .40 and requiring at least a difference of .15 between items loading on the same component.

The Mentoring the Responsible Conduct of Research Scale (MRCR)

Phase 1

Two factors emerged from factor analysis accounting for a cumulative 40% of the variance. As anticipated, items loading on factors 1 (18 items) and 2 (8 items) reflected mentor RCR instruction, e.g. [My mentor] “Made sure I instituted strict procedures to protect confidentiality” and mentor RCR modeling, e.g., [My mentor] “Unfairly assigned authorship of publications or presentations.” Preliminary subscales constructed from these factors yielded alpha coefficients of .89 and .73, respectively, and as predicted positively and significantly correlated with the AWAI-S subscales at p <.001. Correlations for the instruction and modeling items with subscales were respectively: Apprentice r(199) = .64 and .48, p < .001; Identification r(199) = .51 and .55, p < .001]; and Rapport, r(199) = .54 and .65, p < .001. Item analysis of each preliminary subscale helped identify items that might be deleted or modified for Phase 2 testing. For example, item difficulty analyses indicated the weakest items described the mentor as actively engaged in unethical activity or encouraging/permitting students to do so. Corrected item-total correlations suggested that the least frequently endorsed items (e.g., “permitting the student to read confidential grant proposals or journal manuscripts he/she was reviewing”) yielded the lowest discriminability.

Phase 2

Based on the Phase 1 analyses we divided items into an MRCR-I (Instruction) subscale (“My mentor gave me helpful training about...”) and MRCR-M (Modeling) subscale (“My research mentor...”). In addition, to avoid confounding of negative wording with items reflecting mentor RCR modeling, we worded all items positively and added items to the implicit scale that included mentor-protégé responsibilities (e.g. “Was available to discuss questions about research ethics”). Following factor and item analysis, 2 subscales with high levels of inter-item consistency emerged: The 19-item MRCR-Instruction (MRCR-I) and the 9-Item MRCR-Modeling (MRCR-M) yielded alphas of .95 and .90, respectively. Table 2 provides the items maintained for each subscale, their factor loadings, alphas-if-item-deleted, corrected-item-total-scores, item means and standard deviations.

Table 2

MRCR Subscale Items Maintained, Factor Loadings, Means and Standard Deviations, Corrected-Item Total Correlations and Alpha If Item Deleted.

Subscale Coeff. α ItemLoadingMeanSDCorrected Item-Total CorrelationAlpha If Item Deleted
MRCR-Instruction.951. Appropriate informed consent procedures..604.711.11.64.95
2. Avoiding plagiarism in my writing..574.561.22.64.95
3. Preparing an Institutional Review Board (IRB) application.514.451.30.60.95
4. How to protect participant confidentiality..624.621.10.67.94
5. When to disclose confidential information (e.g., child abuse, harm to self or others)..734.421.25.61.95
6. Data sharing with other investigators..744.101.36.70.94
7. Financial or personal conflicts of interest that could bias my research..793.941.32.71.94
8. Potentially harmful multiple relationships with research participants who are also the investigator's students or clients..763.991.28.70.94
9. Federal regulations governing research..683.881.36.68.94
10. American Psychological Association Ethics Code standards..673.861.32.67.94
11. Ensuring participant recruitment is voluntary and non-coercive..634.731.21.73.94
12. Prohibitions against data fabrication..664.661.34.75.94
13. Prohibitions against data falsification..634.621.33.70.94
14. Disguising the identities of research participants when reporting my research..604.651.25.67.94
15. Appropriate collection and storage of data..574.641.29.65.95
16. Accurate reporting of research expenses..674.071.40.67.94
17. Fair and non-coercive participant incentives (i.e., cash payments, lotteries, course credit)..624.781.13.72.94
18. Debriefing procedures..554.481.32.64.95
19. Evaluating research risks and benefits..694.631.22.74.94
MRCR-Modeling.901. Had a positive influence on shaping my professional values as a researcher..644.971.18.73.89
2. Was knowledgeable about ethics..585.050.92.61.89
3. Conducted his/her own research ethically..745.390.80.67.89
4. Was available to discuss questions about research ethics..655.031.13.69.89
5. Supervised my research in an ethical manner..725.211.02.72.89
6. Handled data ownership fairly..815.320.98.69.89
7. Collaborated with other professionals to ensure that I received the support or training I needed to conduct my research..574.661.47.59.90
8. Discussed authorship of publications that might emerge from my research..554.841.35.59.90
9. Assigned (or will assign) authorship on research collaborations fairly..795.241.10.65.89
10. Provided timely and helpful feedback on all aspects of my research project..694.881.30.72.89

Kaiser-Meyer-Olkin Measure of Sampling Adequacy: .918

Bartlett's Test of Sphericity: χ2 = 2766.67, df = 300, p < .001

N = 241

The Responsible Conduct of Research-Department Climate Index (RCR-DC)

Phase 1

The 2 factors emerging from the factor analysis accounted for a cumulative 36% of the variance. The 7 items loading on factor 1 reflected statements describing formal RCR department policies and procedures, e.g., “Students are made aware of policies prohibiting research data falsification and fabrication.” The 11 items loading on factor 2 reflected faculty and student RCR (non)compliance, e.g., “Research funds are misused.” As predicted, subscales derived from factors 1 and 2 were significantly correlated with the Research Training Environment Scale [r(199) = .30, p < .001, and r(199) = .24, p < .001, respectively]. However, these subscales yielded only fair internal consistency, α = .70 and .78, respectively. Item analyses indicated poor levels of difficulty for most items describing faculty as purposefully violating RCR norms. Items yielding low item-total correlations (.18 -.33) appeared to be those with the least variability and lowest frequency, e.g., “Faculty members have a tendency to ignore ethical standards for psychological research”.

Phase 2

Based on the Phase 1 analyses, we sought to strengthen the distinctive contributions of different factor clusters by retaining some items, deleting or modifying others, and following Victor and Cullen (1988), adding items that would demonstrate RCR core principles such as the relative weight given to research versus research ethics (“Getting research completed takes priority over conducting the research ethically”). Thirty-eight items were tested, balanced with respect to positive and negative wording. Factor analyses yielded 15 items loading on factor 1 (representing RCR formal department policies and resources), and 12 loading on factor 2 (reflecting faculty and student RCR behaviors), met these criteria. Separate item analyses were conducted on the two factor loadings. One item, “Graduate research assistants are exploited,” was dropped from the formal policy factor because it was the only item that yielded a higher total alpha-if-item-deleted and it yielded the lowest corrected item-total correlation r = .44, compared with the .55 - .77 range of other items. The resultant RCR-DC Policy (RCR-DC-PY) 15-item subscale yielded an alpha coefficient of .91. Two items were eliminated from the second factor. “Students and faculty are expected to obtain IRB approval for their research,” was the most skewed item, yielding a .34 corrected-item-total-correlation and was the only item to raise the alpha level if deleted. A second item, “Faculty members ignore plagiarism,” was deleted because it was the only item that did not explicitly refer to a research/publication issue and could be confused with non-research related paper writing. The resultant 9-item RCR-DC Practices (RCR-DC-PR) subscale yielded an alpha coefficient of .84. RCR-DC subscale items that were maintained, factor loadings, alpha-if-item-deleted, corrected-total-scores, means and standard deviations are provided in Table 3.

Table 3

RCR-Departmental Climate Subscale Items Maintained, Factor Loadings, Means and Standard Deviations, Corrected-Item Total Correlations and Alpha If Item Deleted

Subscale Coeff. α ItemLoadingMeanSDCorrected Item-Total CorrelationAlpha If Item Deleted
RCR-DC-PY (Policy).9141. Concern for the welfare of research participants is stressed in courses..654.721.15.62.91
2. A major consideration is whether research design protects participants’ autonomy..604.511.07.56.910
3. There is a high level of support for making ethically sound decisions about research..694.880.85.71.91
4. Students and faculty are expected to be familiar with federal regulations governing research..544.791.07.53.91
5. Students and faculty are expected to comply with the APA Ethics Code..435.420.82.46.91
6. Students are made aware of specific rules prohibiting research misconduct..594.571.25.57.91
7. There is a clear policy for handling research ethics complaints..693.931.32.64.91
8. There is a written policy on how to avoid conflicts of interest in research..663.701.26.51.91
9. Students know where to get guidance if they have a research ethics question..734.481.26.68.91
10. Adherence to research ethics standards is carefully monitored.784.181.27.74.90
11. Research ethics policies are effectively communicated to students..814.331.16.78.90
12. Faculty consistently model ethical research practices..604.870.92.63.91
13. Students are encouraged to express any ethical concerns they have about research..704.641.07.67.91
14. High standards of research ethics are as important as high standards of research design..584.981.10.60.91
15. Students and faculty are held personally accountable for the ethical conduct of their research..565.160.89.60.91
RCR-DC-PR (Practices).841. Faculty and students engage in ethically questionable research practices..495.071.11.46.84
2. Research funds are misused..605.220.93.55.83
3. Faculty and students are expected to build up their research resume, regardless of whether or not the study complies with ethical standards..725.181.08.65.82
4. Getting research completed takes priority over conducting research ethically..725.231.01.69.81
5. Research is considered unethical only if someone makes a public complaint..695.440.86.65.82
6. Research productivity that violates ethical standards is rewarded..555.410.93.52.83
7. Research ethics are considered only a matter of personal values..765.220.98.66.82
8. Students and faculty decide for themselves whether research procedures are or are not ethical..645.071.16.51.83
9. Research ethics procedures are guided by the investigator's personal ethics..584.461.25.43.85

Kaiser-Meyer-Olkin Measure of Sampling Adequacy: .918

Bartlett's Test of Sphericity: χ2 = 2766.67, df = 300, p < .001

N = 241

The Responsible Conduct of Research-Student Preparedness Scale (RCR-P)

Phases 1 & 2

The 23 items developed for Phase 1 yielded high and consistent alphas-if-item-deleted (.95). Mean corrected-item-total-correlations were in the acceptable range (.50) as were levels of item difficulty. As predicted, in Phase 1 the mean scale score was significantly and positively correlated with the SERM, r(199) = .63, p < .001. As illustrated in Table 4, item analysis conducted on the same items administered in Phase 2 yielded similarly strong results and a total scale alpha coefficient of .95 .

Table 4

RCR-P [Preparedness] Items, Means, Standard Deviations, Corrected-Item-Total Correlations, Alpha If Item Deleted.

Item (Total scale reliability: α = .95)MeanSDCorrected Item-Total CorrelationAlpha If Item Deleted
1. Maintain research records in a manner consistent with APA ethical standards.4.731.12.70.95
2. Appropriately debrief research participants.4.821.16.66.95
3. Assign appropriate authorship credit for publications4.901.19.69.95
4. Fairly review research proposals or manuscripts submitted by peers for publication.4.381.38.64.95
5. Collaborate with other professionals to implement research and avoid potential disputes.4.591.24.73.95
6. Have adequate familiarity with federal regulations for the protection of human subjects in research.4.581.15.67.95
7. Avoid plagiarism when writing research proposals or manuscripts.5.290.91.63.95
8. Avoid scientific misconduct.5.230.91.73.95
9. Adequately protect confidential research data.5.170.88.63.95
10. Know when it is ethically appropriate to disclose a research participant's confidential information.4.561.17.56.95
11. Protect research participant confidentiality when teaching or speaking at professional meetings.5.200.97.63.95
12. Accurately report research expenditures to institutions and funding sources.4.211.33.64.95
13. Ensure research assistants are properly trained in research ethics.4.481.28.72.95
14. Prevent behaviors that unfairly discriminate against research assistants or participants.4.781.22.79.95
15. Know when it is ethically required to share research data with other investigators.3.961.32.65.95
16. Offer research incentives that are fair and non-coercive.4.851.06.68.95
17. Adequately prepare applications for Institutional Review Board (IRB) approval to conduct research5.031.05.57.95
18. Accurately report research results in public presentations.5.140.97.65.95
19. Identify financial or personal conflicts of interests that could bias my research.4.541.13.62.95
20. Avoid harmful or exploitive multiple relationships with students, research assistants or research participants.4.941.12.62.95
21. Develop informed consent procedures according to the APA's Ethics Code criteria.5.030.97.70.95
22. Protect students, client/patients or other subordinates from being coerced into participating in research.5.031.05.70.95
23. Know when it is ethically appropriate to dispense with informed consent.4.311.24.60.95

The Responsible Conduct of Research- Field Integrity Scale (RCR-FI)

The 13-item RCR-FI was introduced in Phase 2. Item analysis indicated that 1 item (“Psychologists tend to disregard ethical dilemmas that arise in their research”) would raise the alpha level if deleted, and yielded a negative corrected item total, and was highly skewed, producing the least endorsements. The resulting 12-item scale (see Table 5) yielded an alpha coefficient of .89.

Table 5

RCR-FI [Field Integrity] Item Means, Standard Deviations, Corrected-Item-Total Correlations, Alpha If Item Deleted

Item (Total scale reliability: α = .89)MeanSDCorrected Item-Total CorrelationAlpha If Item Deleted
1. It is common practice for research psychologists to consider and resolve ethical problems.5.030.99.57.88
2. Psychologists consider the ethical implications of their research.4.990.85.61.88
3. Research that was conducted unethically is not accepted for publication in psychology journals.4.451.32.44.89
4. There are adequate safeguards to ensure psychologists engage in ethical research practices.4.401.11.58.88
5. Conducting research ethically is valued in the field of psychology.5.170.92.66.87
6. The public can trust psychologists not to fabricate data.4.661.00.51.88
7. Psychologists are highly invested in conducting research ethically.4.810.91.76.87
8. Psychology faculty members model the ethical conduct of research.4.830.87.73.87
9. Psychology graduate students receive adequate training in research ethics.4.111.20.56.88
10. The rights of participants in psychological research are adequately protected.4.970.71.59.88
11. The field of psychology encourages its members to conduct ethical research.5.330.76.66.88
12. Conducting research ethically is the norm in psychology.4.980.77.68.88

Construct Validity

We evaluated the construct validity of the RCR scales emerging from the Phase 2 analysis through evaluation of hypothesized relationships among RCR scale scores and between RCR scale scores and student and program characteristics. As illustrated in Table 6, RCR scales were positively correlated with one another, and also positively correlated with the AWAI-S Rapport scale and single item responses for satisfaction with mentoring, department ethics climate, and graduate program. The results support the assumption that, overall, the degree to which mentors provide positive RCR socialization to their students is related to support for RCR provided by the department, and both of these factors, in turn, are related to students’ satisfaction with their graduate experience, their perceived preparedness to implement RCR procedures and their evaluation of the RCR integrity of the field of psychology.

Table 6

Correlations Among RCR Scales and Between RCR Scales and Student, Program, and Mentor Characteristics (N = 241).

MRCR† InstructionMRCR†† ModelingRCR-DC PolicyRCR-DC PracticesRCR-PRCR-FI
MRCR-Instruction--
MRCR-Modeling.72***--
RCR-DC Policy.47*** (.25***).44*** (.17**)--
RCR-DC Practices.16* (-.01).23*** (.16**).53***--
RCR-Preparedness.55*** (.36***).46*** (.11).64***.41***--
RCR-Field Integrity.36*** (.17**).35*** (.14*).65***.53***.54***--
AWAI-S Rapport Subscale.50*** (.05).66*** (.50***).32***.23***.34***.23***
Yr. Anticipate Receiving Degree-.14* (-.09)-.11 (-.01)-.02-.05-.24***-.04
Primary Career Goal.04 (-.03).08 (.08)-.01-.05.14*.04
# Publications.13* (.03).16* (.09)-.11-.07.21***-.01
Completed an Ethics Course.07 (.10)-.01 (-.08).14*-.02.08.07
Department Ethics Requirement.02 (.05)-.01 (-.04).18**.14*.03.14*
Years w/Mentor.05 (.05).02 (-.02)-.15*-.10.13*-.17**
Mentor Assigned-.02 (-.01)-.02 (-.01)-.07.06-.02-.05
Mentor Satisfaction.55*** (.26).57*** (.29***).39***.20**.43***.38***
Satisfaction RCR Dept. Climate.35*** (.09).40*** (.24***).53***.31**.41***.44***
Satisfaction Grad Program.38*** (.16).39*** (.18**).34***.16*.34***15*
†MRCR-Instruction partial correlations controlling for MRCR-G
††MRCR-Modeling partial correlations controlling for MRCR-C
*p ≤ .05
**p ≤ .01
***p ≤ .001

We expected that the number of student publications would be associated with higher levels of positive RCR mentoring, since student authorship is associated with greater participation and mentor supervision in research design, including the design and implementation of ethical procedures. This prediction was confirmed. We also assumed and confirmed that students who had taken an ethics course and whose department required one would report significantly greater scores on the RCR-DC Policy scale. A required ethics course was also significantly associated with higher responses on the RCR-DC Practices scale, suggesting that the ethical sensitivity of both faculty and students may be heightened when an ethics course is required of all students. Similarly, students whose program had a required ethics course had more positive attitudes toward the RCR integrity of the discipline.

As predicted, the closer students were to receiving their doctorate and the greater the number of student publications they reported, the more likely they would report they were prepared to conduct research responsibly as measured by the RCR-P. As might also be expected, how close they were to receiving their doctorate was not significantly related to any of the RCR mentoring, department climate, or field integrity scores. Contrary to expectations, students who identified research as their primary or secondary career goal and those who reported completing a graduate course in ethics did not report higher levels of RCR preparedness.

We predicted that RCR mentoring and department climate scores would significantly and independently contribute to the variance in RCR preparedness and RCR attitudes toward field integrity. To test these predictions, multiple regressions using the enter method were conducted (see Tables 7 and ​and8).8). Based on the bivariate correlations, predictor variables for RCR-P scores included year of anticipated degree, number of student publications, and research as a career goal (Block 1), the AWAI-S Rapport, the MRCR-Instruction and MRCR-Modeling (Block 2) and the RCR-DC-Policy and RCR-DC-Practices subscales (Block 3). A significant model emerged with R squares accounting for 10%, 36% and 59% of the variance for blocks 1, 2, and 3, respectively, and significant F change scores for each block. Beta values indicated that along with the student characteristics both MRCR-Instruction and RCR-DC-Policy independently contributed to the variance in students’ RCR preparedness.

Table 7

Student Characteristics, Mentor Rapport, and MRCR and RCR-DC Subscales Regressed onto RCR-Preparedness Scores

R2 β t p
Block 1.10
    Q8 Research as primary career goal.142.23.03
    Q11 Year anticipate receiving or completed doctoral degree-.23-3.56.00
    Q13 Number of publications on which you are co-author.132.04.04
Block 2.36
    Q8 Research as primary career goal.112.12.04
    Q11 Year anticipate receiving or completed doctoral degree-.16-3.00.00
    Q13 Number of publications on which you are co-author.081.39.17
    AWAI-S Rapport.04.61.54
    MRCR-Instruction.435.68.00
    MRCR-Modeling.09.97.33
Block 3.59
    Q8 Research as primary career goal.122.65.01
    Q11 Year anticipate receiving or completed doctoral degree-.16-3.71.00
    Q13 Number of publications on which you are co-author.184.01.00
    AWAI-S Rapport.02.30.77
    MRCR-Instruction.274.22.00
    MRCR-Modeling-.04-.55.58
    RCR-DC Policy.498.53.00
    RCR-DC Practices.102.03.04

Table 8

Analysis of Mentor Rapport and MRCR and RCR-DC Subscales Regressed onto RCR-Field Integrity Scores

R2 β t p
Block 1.15
    AWAI-S Rapport Subscale-.01-.10.92
    MRCR-Instruction.222.58.01
    MRCR-Modeling.191.90.06
Block 2.48
    AWAI-S Rapport Subscale-.06-.99.32
    MRCR-Instruction.081.13.26
    MRCR-Modeling.07.84.40
    RCR-DC Practices.467.33.00
    RCR-DC Policy.274.69.00

Based on the bivariate correlations only the AWAI-S and MRCR subscales (Block 1) and RCR-DC subscales (Block 2) were entered as predictors of RCR-FI scores. A significant model emerged with R squares accounting for 15% and 48% of the variance for blocks 1 and 2, respectively, and significant F change scores for each block. When Block 1 was entered, standardized Beta coefficients indicated that the MRCR-Instruction continued to make a significant independent contribution to the variance when other predictors were held constant. However, when the RCR-DC scores were entered, they, but not the mentoring scores, yielded significant Betas.

Exploratory Analysis of Student, Faculty and Program RCR Differences

MANOVAS and bivariate correlations were conducted to determine whether student (e.g. gender, years working with mentor, number of ethical challenges confronted in research), mentor (gender, professional status) or program (number of faculty, percent ethnic minority faculty and students, number of students admitted each year, number of doctoral programs offered) characteristics yielded significant mean differences or relationships with the RCR scales that were not predicted by the literature. None of these analyses yielded significance.

Discussion

The RCR scales constructed and assessed in this study were developed to contribute to the empirical evaluation of RCR socialization in graduate programs of psychology. The two-phase process allowed us to evaluate item content validity, provide evidence for the multidimensionality of RCR mentoring and department climate, increase internal scale consistency, and demonstrate concurrent and construct validity. The scales and subscales that emerged yielded good psychometric properties and support the use of these scales for future empirical investigations of how research ethics values are socialized and internalize in graduate psychology training programs.

As anticipated from the literature, factor analyses indicated that research ethics socialization of graduate psychology students occurs explicitly through mentor guidance and formal department policies and implicitly through mentor modeling and RCR practices among department faculty and students. Correlations between initial versions of the MRCR, RCR-DC and RCR-P with existing measures of research mentoring, department climate and student efficacy support the concurrent validity of the measures, and further suggest that socialization and internalization of research and research ethics values and practices are distinct but inter-related aspects of psychology students’ graduate education. Associations between the RCR measures and items tapping student satisfaction indicate that higher levels of RCR mentoring and department climate raise students’ overall satisfaction with their graduate education.

Our prediction that students whose mentors were selected rather than assigned by the department, those with more years with their mentor and with more publications would be more likely to rate their mentor higher in both RCR instruction and modeling was unsupported. While unanticipated, it suggests that factors associated with components of effective RCR mentoring as measured by the MRCR may be distinct from how a mentor is evaluated as contributing to a protégé's knowledge of research design and support for research career goals. Along similar lines, as anticipated, scores on the RCR-Preparedness scale were positively correlated with more student-authored publications and to completion of the doctorate; however, these two variables were not associated with RCR-Field Integrity scores. This further suggests that RCR measures we developed are tapping distinct research ethics relevant dimensions of graduate education.

Contrary to expectations, neither students who identified research as their primary or secondary career goal nor those who reported completing a graduate course in ethics reported higher levels of RCR preparedness. We speculate that one reason for this pattern of results may be related to differences in ethics course requirements across fields within the discipline. To satisfy APA program accreditation requirements, students in clinical, counseling and school psychology programs uniformly receive some type of formal ethics education irrespective of whether they have a career interest in research, while programs focused exclusively on research are not eligible for APA accreditation and usually do not offer formal ethics courses. In future studies larger samples sizes will be needed to tease apart career interest in research from ethics course requirements and offerings between practitioner, science-practitioner, and science programs.

Supporting the value of the RCR scales to examining graduate student research ethics socialization, correlations among new scales and demographic characteristics indicated that RCR mentoring and department climate measures were, in many instances, more likely than other student, mentor and program characteristics to generate significant relationships with students’ reports of their RCR preparedness and their attitudes toward the research integrity of the field. Multiple regression analyses further demonstrated the independent contribution of responses on the RCR mentoring and department climate scales to RCR preparedness and attitudes toward the discipline. In this regard, the pattern of beta values suggests that at least for scores on the RCR-FI [Field Integrity], the contribution of RCR mentoring as measured through the MRCR subscales may be mediated by the RCR department climate (Baron & Kenny, 1986). Continued evidence of mediating effects provided by future administration of the RCR instruments in larger national samples of psychology graduate students would lend support to the proposition that given the variability in RCR mentoring, departments can offer consistent and effective training that can compensate for instances of poor RCR mentoring (NAS, 2002).

Sampling Strengths and Limitations

Demographic data suggest that the web-based recruitment methodology was successful in obtaining representative samples of students by gender, research methods, career goals, and level of graduate education; mentor gender, professional status and years mentoring student; program mission, size, geographic location and faculty and student ethnic composition. The sample sizes of Phases 1 and 2 were sufficient as a first test of the validity and internal consistency of the scales; however, some limitations should be noted when considering administration of these instruments. First, while the proportion of ethnic minority psychology graduate students paralleled those of the field and exploratory analyses revealed no significant differences between students identifying as non-Hispanic white or as a member of a specific ethnic minority population, the sample size was too small to draw definitive conclusions about the validity of these measures for ethic minority student populations. Similarly, the female-to-male ratio was somewhat higher than that in the field. Future research administered to larger samples will provide a clearer picture of whether gender or ethnicity influences student responses. In both phases, the scales skewed positively. While the anonymous, voluntary and web-based nature of the data collection would be expected to lower social desirability, there is no way to know whether the more positive skewing scales reflected the full diversity of student opinions. In addition, given the web-based nature of the recruitment and testing, it is difficult to estimate with any precision the proportion of students who chose not to participate. Finally, the self report methods used to gather information on RCR mentor and program characteristics and student RCR preparedness are always vulnerable to inaccurate response rates based on poor recall or perceived social desirability.

Conclusions

An outcome of the socialization process in graduate education is the absorption and internalization of the RCR values eschewed by the graduate institution (Anderson & Louis, 1994; NAP, 2002). Experienced mentors assist students as they are socialized into a community's practices and make those practices part of their subjective reality (Fisher, Wertz, & Goodman, 2009). Preparing students to conduct research responsibly and to see themselves as members of a discipline that values research integrity also requires an ethical climate at the department level that provides clear and unambiguous policies regarding scientific integrity, encourages RCR curriculum development, and fosters ethical behavior in the actual conduct of research by faculty and students.

Psychology as a discipline has a long history of valuing and promoting the ethical conduct of research. However, there has been little empirical attention to how these values are transmitted to psychology graduate students. There appears to be a growing consensus that ensuring RCR practices in future generations of scientists require that graduate programs foster in their students awareness and understanding of the standards, regulations, and ethical values of the scientific community (NAS, 1997). The paucity of suitable methods for the assessment of RCR socialization and internalization in science programs and psychology specifically has been an impediment to critical evaluation of current research ethics curricula. The RCR instruments developed and tested in the study reported in this article are a beginning step in meeting the need for the development and application of adequate measures to evaluate educational factors that promote integrity in psychological research.

Acknowledgments

This research was supported by grant #1 R01 NS052877-01 from the National Institute of Neurological Disorders and Stroke, PI Celia B. Fisher. We thank Mary Brabeck, Leslie Burton, James DuBois, Michael Kalichman, Gerald Koocher, Georgiana Tryon, and Frederick Wertz for their expertise.

References

  • American Psychological Association . Ethical standards of psychologists. Author; Washington, DC: 1953. [Google Scholar]
  • American Psychological Association Ad Hoc Committee on Ethical Standards in Psychological Research . Ethical principles in the conduct of research with human participants. APA; Washington, DC: 1973. [Google Scholar]
  • American Psychological Association Ethical principles of psychologists and code of conduct. American Psychologist. 1992;47:1597–1611. [Google Scholar]
  • American Psychological Association Ethical principles of psychologists and code of conduct. American Psychologist. 2002;57:1060–1073. [Abstract] [Google Scholar]
  • Anderson MS, Louis KS. The graduate student experience and subscription to the norms of science. Research in Higher Education. 1994;35:273–299. [Google Scholar]
  • Anderson MS, Louis KS, Earle J. Disciplinary and departmental effects on observations of faculty and graduate student misconduct. Journal of Higher Education. 1994;65:331–350. [Google Scholar]
  • Ashforth BE. Climate formation, issues and extensions. Academy of Management Review. 1985;10:837–847. [Google Scholar]
  • Baron RM, Kenny DA. The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology. 1986;51:1173–1182. [Abstract] [Google Scholar]
  • Baumrind D. Some thoughts on ethics of research: After reading Milgram's “Behavioral study of obedience.” American Psychologist. 1964;26:887–896. [Google Scholar]
  • Bird SJ. Mentors, advisors, and supervisors: their role in teaching responsible research conduct. Science and Engineering Ethics. 2001;7:455–468. [Abstract] [Google Scholar]
  • Brown S, Kalichman MW. Effects of training in the responsible conduct of research: A survey of graduate students in experimental sciences. Science & Engineering Ethics. 1998;4:487–498. [Abstract] [Google Scholar]
  • Clark RA, Harden SL, Johnson WB. Mentor relationships in clinical psychology doctoral training: Results from a national survey. Teaching of Psychology. 2000;27:262–268. [Google Scholar]
  • Council of Graduate Schools . The role and nature of the doctoral dissertation: A policy statement. Author; Washington, D.C.: 1991. [Google Scholar]
  • DHHS (Department of Health and Human Services) [26 August 2006];Public Welfare: Code of Federal Regulations: Protection of Human Participants. 45CFR46. 2005 Available: http://www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.htm.
  • Eisen A, Berry RM. The absent professor: Why we don't teach research ethics and what to do about it. American Journal of Bioethics. 2002;2:38–49. [Abstract] [Google Scholar]
  • Fisher CB. Decoding the Ethics Code: A Practical Guide for Psychologists. Sage Publications; Thousand Oaks, CA: 2003. [Google Scholar]
  • Fisher CB, Tryon W. Ethics in applied developmental psychology: Emerging issues in an emerging field. Ablex Publications; Norwood, NJ: 1990. [Google Scholar]
  • Fisher CB, Wertz FJ, Goodman SJ. Mentoring the responsible conduct of social science research. In: Mertens DM, Ginsberg P, editors. Handbook of Social Research Ethics. Sage Publications; Thousand Oaks, CA: 2009. pp. 550–565. [Google Scholar]
  • Fisherman JA, Galguera T. Introduction to test construction in the social and behavioral sciences: A practical guide. Rowman & Littlefield; Lanham, MD: 2003. [Google Scholar]
  • Fryer-Edwards K. Addressing the hidden curriculum in scientific research. American Journal of Bioethics. 2002;2:58–59. [Abstract] [Google Scholar]
  • Gelso CJ. On the making of a scientist-practitioner: A theory of research training in professional psychology. Training and Education in Professional Psychology. 2006;S:3–16. [Google Scholar]
  • Gelso CJ, Mallinckrodt B, Judge AB. Research training environment, attitudes toward research, and research self-efficacy: The revised research training environment scale. The Counseling Psychologist. 1996;24:304–322. [Google Scholar]
  • Gelso CJ, Raphael R, Black SM, Rardin D, Skalkos P. Research training in counseling psychology: Some preliminary data. Journal of Counseling Psychology. 1983;30:611–614. [Google Scholar]
  • Gilbert IA. Dimensions of same-gender student-faculty role model relationships. Sex Roles. 1985;12:111–123. [Google Scholar]
  • Hafferty FW, Franks R. The hidden curriculum, ethics teaching, and the structure of medical education. Academic Medicine. 1994;69:861–871. [Abstract] [Google Scholar]
  • Haney C, Banks C, Zimbardo P. Interpersonal dynamics in a simulated prison. International Journal of Criminology and Penology. 1973;1:69–97. [Google Scholar]
  • Hoffer TB, Welch V, Jr., Webber K, Williams K, Lisek B, Hess M, Loew D, Gusman-Barron I. Doctorate Recipients from United States Universities: Summary Report 2005. National Opinion Research Center; Chicago: 2006. [Google Scholar] (The report gives the results of data collected in the Survey of Earned Doctorates, conducted for six federal agencies, NSF, NIH, USED, NEH, USDA, and NASA by NORC).
  • Hollingsworth MA, Fassinger RE. The role of faculty mentors in the research training of counseling psychology doctoral students. Journal of Counseling Psychology. 2002;49:324–330. [Google Scholar]
  • Humphreys L. Tearoom trade: Impersonal sex in public places. Aldine; New York: 1975. [Abstract] [Google Scholar]
  • Johnson WB, Nelson N. Mentor-protégé relationships in graduate training: Some ethical concerns. Ethics & Behavior. 1999;9:189–210. [Google Scholar]
  • Kahn JH, Gelso CJ. Factor structure of the Research Training Environment Scale-Revised: Implications for research training in Applied Psychology. The Counseling Psychologist. 1997;25:22–37. [Google Scholar]
  • Kahn JH, Miller SA. Measuring global perceptions of the research training environment using a short form of the RTES-R. Measurement and Evaluation in Counseling and Development. 2000;33:103–119. [Google Scholar]
  • Kahn JH, Scott NA. Predictors of research productivity and science-related career goals among counseling psychology doctoral students. The Counseling Psychologist. 1997;25:38–67. [Google Scholar]
  • Kalichman MW. Responding to challenges in educating for the responsible conduct of research. Academic Medicine. 2007;82:870–875. [Abstract] [Google Scholar]
  • Keith-Spiegel P, Koocher GP. Ethics in psychology: Professional standards and cases. First Edition Random House; NY: 1985. [Google Scholar]
  • Kitchener KS. Psychologists as teacher and mentor: Affirming ethical values throughout the curriculum. Professional Psychology: Research & Practice. 1992;213:190–195. [Google Scholar]
  • Louis KS, Anderson MS, Rosenberg L. Academic misconduct and values: the department's influence. Review of Higher Education. 1995;18:393–422. [Google Scholar]
  • Magnus P, Kalichman M. Mentoring. [July 6, 2008];University of Ariona Program in Research Integrity Monthly Newsletters. 2007 7:1–4. from http://www.vpr.arizona.edu/system/files/PRIEnewsletter-7+3_2.pdf. [Google Scholar]
  • Mallinckrodt B, Gelso CJ, Royalty GM. Impact of the research training environment and counseling psychology students’ Holland personality type on interest in research. Professional Psychology: Research and Practice. 1990;21:26–32. [Google Scholar]
  • Motta MM. Mentoring the mentors: The Yoda factor in promoting scientific integrity. American Journal of Bioethics Online Peer Commentaries. 2002;2:w1–w2. [Abstract] [Google Scholar]
  • NAS . Responsible science: Ensuring the integrity of the research process. National Academies Press; Washington, D.C.: 1992. [Abstract] [Google Scholar]
  • NAS . Advisor, Teacher, Role Model, Friend: On Being A Mentor to Students in Science and Engineering. National Academies Press; Washington, D.C.: 1997. [Google Scholar]
  • NAS . Integrity in Scientific Research: Creating an Environment that Promotes Responsible Conduct. National Academy Press; Washington, D.C.: 2002. [Abstract] [Google Scholar]
  • Office of Research Integrity (ORI) PHS Policy on Instruction in the Responsible Conduct of Research (RCR) U.S. Department of Health and Human Services; Washington, D.C.: 2000. Available from: http://ori.dhhs.gov/html/programs/finalpolicy.asp. [Google Scholar]
  • Public Health Service (PHS) Policy on instruction in responsible conduct in research. 2000 Available from: http://ori.dhhs.gov/html/programs/congressionalconcerns.asp.
  • Public Health Service (PHS) Responsible conduct in research education policy suspended. 2001 Available from: http://ori.dhhs.gov/html/programs/congressionalconcerns.asp.
  • Royalty GM, Gelso CJ, Mallinckrodt B, Garrett KD. The environment and the student in counseling psychology: Does the research training environment influence graduate students’ attitudes toward research? Counseling Psychologist. 1986;14:9–30. [Google Scholar]
  • Sales BD, Folkman S. Ethics in research with human participants. American Psychological Association; Washington, DC: 2000. [Google Scholar]
  • Schlosser LZ, Gelso CJ. Measuring the working alliance in advisor-advisee relationships in graduate school. Journal of Counseling Psychology. 2001;48:157–167. [Google Scholar]
  • Sieber JE, editor. The ethics of social research: Surveys and experiments. Springer-Verlag; New York: 1982. [Google Scholar]
  • Smith MB. Some perspectives on ethical/political issues in social science research. Personality & Social Psych Bulletin. 1976;2:445–453. [Abstract] [Google Scholar]
  • Steneck NH. ORI Introduction to the Responsible Conduct of Research. Department of Health and Human Services; 2001. Available from: http://ori.dhhs.gov/publications/ori_intro_text.shtml. [Google Scholar]
  • Steneck NH. The history, purpose, and future of instruction in the responsible conduct of research. Academic Medicine. 2007;82:829–834. [Abstract] [Google Scholar]
  • Stern JE, Elliott D. The ethics of scientific research: A guidebook for course development. University Press of New England; Hanover: 1997. [Google Scholar]
  • Swazey JP, Anderson MS. Mentors, advisors, and role models in graduate and professional education. Association of Academic Health Centers; Washington, D.C.: 1996. [Google Scholar]
  • Swazey JP, Anderson MS, Louis KS. Ethical problems in academic research. American Scientist. 1993;81:542–553. [Google Scholar]
  • Tryon GS. School psychology students’ beliefs about their preparation and concern with ethical issues. Ethics & Behavior. 2002;11:375–394. [Google Scholar]
  • Victor B, Cullen JB. The organization basis of ethical work climates. Administrative Science Quarterly. 1988;33:101–125. [Google Scholar]
  • Wilson PF, Johnson WB. Core virtues for the practice of mentoring. Journal of Psychology & Theology. 2001;29:121–130. [Google Scholar]
  • Wright DE, Klomparens K. Ethical research. Michigan State University: Research Integrity. 1998;1:2–5. [Google Scholar]

Full text links 

Read article at publisher's site: https://doi.org/10.1080/10508420902886726

Read article for free, from open access legal sources, via Unpaywall: https://europepmc.org/articles/pmc3634717?pdf=renderUnpaywall

Citations & impact 

Impact metrics

7
Citations
Jump to Citations

Citations of article over time

Alternative metrics

Altmetric item for https://www.altmetric.com/details/46105657
Altmetric
Discover the attention surrounding your research
https://www.altmetric.com/details/46105657

Smart citations by scite.ai
Smart citations by scite.ai include citation statements extracted from the full text of the citing article. The number of the statements may be higher than the number of citations provided by EuropePMC if one paper cites another multiple times or lower if scite has not yet processed some of the citing articles.
Explore citation contexts and check if this article has been supported or disputed.
https://scite.ai/reports/10.1080/10508420902886726

Supporting
Mentioning
Contrasting
2
15
0

Article citations

Go to all (7) article citations

Similar Articles 

To arrive at the top five similar articles we use a word-weighted algorithm to compare words from the Title and Abstract of each citation.

Funding 

Funders who supported this work.

NINDS NIH HHS (2)