Abstract

Introduction

As institutional review boards (IRBs) face criticism for being over-burdened and under-resourced, the time commitment of their highly trained and relatively expensive members places considerable demands on institutional resources.¹ Some surveys have reported that IRBs average approximately 15 members, with one study reporting a high of 44 members on one IRB.² In contrast, federal regulations require only five members, including a scientist, a non-scientist, and a member unaffiliated with the institution.

The practice of maintaining large IRBs raises the question of whether membership that extends much beyond the minimal regulatory requirements is necessary. Of course one reason for an extensive membership is the need for reviewers whose technical expertise addresses specific topics of review. However, IRBs need not have all members present and can call on specialized reviewers to assist in review of specific protocols, potentially reducing the importance of having diverse expertise present at all board meetings. Moreover, a large standing membership requires a substantial commitment of resources. The proportion of salary, overhead, other services, and travel costs associated with board member review time has been estimated conservatively at 22% of total IRB costs, with high-volume IRBs estimated to cost well over $1 million annually.³ Thus, requiring fewer members at meetings could result in considerable cost savings. Although these expenses may be hidden because IRB participation is often included under more general duties, they remain real costs.

Given that IRB panels are expensive, yet have the option of calling on outside experts, the question is whether a membership of fifteen or even larger (as were 50% of the panels we studied) is helpful. One way that a large membership might be justified is if member interactions at meetings make a significant contribution to IRB discussions. The empirical questions are how often, how extensively, and from which roles such interactions occur. Anecdotal data suggest that chairs and reviewers assigned for each protocol may make the greatest contributions to discussions, but to date there have been no systematic data describing how frequently anyone other than an assigned reviewer or the committee chair participates in protocol discussions. If “ancillary” participants rarely speak, then it is difficult to argue that these participants play an important role in IRB deliberations. We use data from a unique observational study of IRBs in major academic medical centers to examine this question.

Methods

Data Collection

We observed and recorded discussions and participation patterns at IRB meetings at 10 academic medical centers. The sites were recruited from among the 25 largest medical centers receiving NIH funds in 2004 – a group chosen because such medical centers do much of the clinical research in the U.S. and because, as larger centers, they should have sufficient resources for proper implementation of research review. At each site, we studied two panels that reviewed new or re-submitted general applications, observing and audio-taping one meeting of each panel. Two sites were excluded from this analysis because of concerns about our accuracy in recording the presence of IRB members who did not speak during the meeting. The median number of new or resubmitted proposals discussed at each meeting was 5 (range 3 to 10). The panels we observed included between 8–21 members in attendance, with a median of 16. A detailed description of the methods can be found elsewhere.⁴

This study received IRB approval at each of the sites involved, as well as at the home institutions of the principal investigator and each co-investigator. All participants at the observed IRB meetings were asked for their written consent to be audiotaped and interviewed; 196 of 226 potential subjects consented (159 IRB members and 37 staff). In keeping with our agreement with the participating IRBs, the contents of statements by IRB members who declined to participate were deleted from the data although the observation that they spoke was not. Seven protocols were excluded because the primary reviewers of those protocols did not consent to participate. At the eight sites, we observed and recorded discussions of 70 new protocols and 11 previously deferred protocols, for a total of 81. Table 1 presents data on the research focus and status of the 81 protocols.

Table 1

Characteristics of Protocols Reviewed by the IRBs

Characteristic	n	%
Field of Medicine
Infectious Diseases	9	11.1
Oncology	19	23.5
Neurology/Psychiatry	8	9.9
Circulatory System	6	7.4
Other	39	48.2
Total	81	100.0
Review Status
New	70	86.4
Deferred	11	13.6
Total	81	100.0
N of Sites
Single-site	42	51.9
Multi-site	39	48.2
Total	81	100.0
Therapeutic/Non Therapeutic
Therapeutic	44	54.3
Non Therapeutic	37	45.7
Total	81	100.0
Study Type
Observational	18	22.2
Intervention	62	76.5
Neither	1	1.2
Total	81	100.0
Study Design
Phase I	12	14.8
Phase II	22	27.2
Phase III	22	27.2
Feasibility	1	1.2
Laboratory	7	8.6
Survey/Interview	1	1.2
Other	16	19.8
Total	81	100.0

Open in a separate window

Panel meetings were transcribed and redacted of any information that might identify the site, PI, or protocol. A detailed codebook was developed on the basis of observations from a pilot study, and modified using data collected from the early sites (codebook available on request from the authors). Two investigators coded the meetings separately and then compared results to resolve disagreements. Further disagreements were resolved by review with the principal investigator. Here we report primarily on two measures of participation at IRB meetings: the number of speaking turns by the persons in given roles and the number of words spoken in those speaking turns.ⁱ Speaking turns were identified as beginning with any statement that could be assigned to a specific individual and continuing until an interruption from another speaker.

To assess the level of participation of panel members according to their role in reviewing a particular protocol, we divided participants into 7 roles: chair/non-reviewer, chair/reviewer, staff/non-reviewer, staff/reviewer, primary reviewer, secondary or tertiary reviewer, and all other non-reviewers. An individual’s role often changed from protocol to protocol. For each role we counted the number of times the member(s) in that role spoke during each protocol discussion (speaking turns), and the number of words in their statements. We averaged both turns and number of words by the number of people in each role for each protocol review.

Results

The degree of participation by IRB panel members is shown in Table 2. Panel chairs, as reviewers and non-reviewers, played very large roles, measured both by speaking turns and word counts. Primary reviewers and chair/reviewers spoke the most words. However what is striking in these results is the relatively minor degree of participation of other members when they were not reviewers. They averaged only 1.8 speaking turns and fewer than 25 words per protocol review. This is further illustrated by the frequency of recommendations made by the primary reviewer and secondary or tertiary reviewers compared to the non-reviewers (see Table 3). Primary reviewers averaged 3.9 speaking turns related to recommendations per protocol compared to non-reviewing members who averaged 0.2 per protocol. The overall pattern of discussion was characterized by lengthy introductions by reviewers followed by very short exchanges with other members (Table 4).

As shown in Figure 1, the median number of IRB members other than the chair and designated reviewers who spoke at all during a protocol review was 2. For 14.8% of protocol reviews, there was no speaker other than a reviewer and the chair. In 62.9% of protocol discussions, 50% or more of the members remained completely silent. Although 2 or more non-reviewers spoke during 66.7% of protocol reviews, their comments were typically brief and emphasized points made by previous speakers. At any given meeting, between 6.7 and 44.4% of all members said nothing, with 23.9% of members at all the meetings we observed remaining silent throughout the entire meeting. Moreover, the larger the meeting, the greater the percentage of members who did not participate in the discussion. The correlation between the number of people attending the meeting and the percentage who did not speak is .73 (Pearson’s r).

Open in a separate window

Figure 1

Number of members other than chairs, reviewers and staff who spoke during protocol reviews

Discussion

Most protocol reviews at the IRB meetings we observed consisted of long descriptions of the protocol by designated reviewers, especially the primary reviewer, followed by a few short exchanges with other members and a significant number of contributions from the chair. From the initial introduction of protocols to the final vote, reviewers joined chairs in leading the discussions. The other IRB members formed a “silent majority,” who often left the substantive work of research review to their panel chairs and designated reviewers. Many IRB members maintained this pattern across the entire meeting.

Although we began with no preconceptions about the implications of the degree of IRB members’ participation on cost and efficiency, our observations indicate that many IRBs may simply be larger than they need to be. Most protocol reviews involve only a small number of members, and non-reviewing members usually do not play an active role in IRB deliberations. Meetings where many members say nothing or contribute little are consequently inefficient and expensive. Charging review fees or incorporating IRB costs into institutional overhead may offset some part of these costs, but practices that require large numbers of members to attend every meeting appear to be a waste of institutional resources. Although our observations do not point to a “correct” number of participants, they do suggest the need to rethink the size of IRBs.

Our findings on the degree of participation and its relationship to the size of the IRB are consistent with classic reports from the business and sociological literatures that larger groups hinder individual participation and lead to dominance by a small number of speakers.⁵ Large groups appear to be particularly prone to “groupthink,” with most members failing to analyze the issues for themselves in the presence of leaders who state their opinions at the beginning of meetings and do not encourage group participation. The fact that a smaller percentage of members participated as the number of members at the meeting grew larger suggests the kind of passivity consistent with the groupthink model. The consequent waste of individual time and resources challenges the current approach to IRB review.

Conclusions

If large numbers of members contribute to the cost of IRB review without substantially altering the outcome, it may be necessary to refine our concept of the appropriate size of review panels. Consistent with the proposed changes to the Common Rule that seek efficiency and reduction of IRB workload, proposals that address the size of IRB meetings are both timely and cost-conscious. Quality reviews may be achieved by identifying outside specialists to review specific protocols and convening much smaller meetings. If the specialists are not local, reviews could be conducted by telephone or submitted by e-mail. The use of outside reviewers, selected because their expertise matches the requirements of a specific protocol, would allow IRB chairs to function in part like journal editors, identifying the appropriate outside experts to contribute to smaller review panels. Indeed, the example of a journal editorial board that bases its decisions on input from expert reviewers but which includes few members who actually participate in publication decisions, may be a more appropriate model for IRB review.

Acknowledgments

Footnotes

References