Introduction

Many public funded research project documents end up sealed in repositories together with unpublished research material on processes and outcomes. Knowledge gained in such circumstances is of limited use precisely because the tensions and contingencies which accompany these processes remain invisible. The idea underlying e-Portfolio is to develop a platform that brings together a range of practice-based design case studies and to assemble all of the materials associated with those studies, including any that relate to the actual research practices involved in their pursuit. By documenting case studies in this way, it makes it possible to do a comparative analysis between them and to inspect the values that shaped their realization. Our work on the e-Portfolio is also motivated by an interest in sharing research results. Researchers across various groups who have similar interests or who work in similar fields, can benefit from former projects and add new insights to enable further comparison and reflection. This resonates with the public interest in data sharing that aims to accelerate knowledge generalization and bring the results of public-funded projects back to the public. However, public and personal interests are sometimes in tension (Friedman et al. 2013) and can involve the need for careful handling of qualitative material and its peculiarities.

Another part of the e-Portfolio vision is to bridge interdisciplinary divides. A prominent requirement in interdisciplinary cooperation is the establishment of a “shared” language, this was pointed out in the context of CSCW by Grudin (1994). This requires an approach that fosters a common understanding beyond just written papers, thereby building a bridge between researchers and practitioners. Value Sensitive Design expresses a similar concern with supporting the assembly of rich descriptions of social practices and design work and tracking long-term appropriation and any resultant changes in social practice to allow for the sharing of data across different groups of stakeholders, as in Walldius et al.’s (2005, 2015) discussion of the UsersAward program. There, the interest was in supporting ongoing software development between trade unions, industry practitioners and researchers in Scandinavia. There, too, the development of a shared language was highlighted as an important issue to be resolved. Walldius et al.’s work built upon earlier work with similar concerns in the DEMOS project (Ehn and Sandberg 1983), which is widely acknowledged to be a cornerstone of the Scandinavian IT design tradition.

Methodological background

Our approach to the development of e-Portfolio is predicated on Grounded Design (Rohde et al. 2017; Wulf et al. 2018). Three methodological perspectives are relevant in Grounded Design: (1) ethnographic research with a specific focus on design and IT appropriation; (2) participatory design, where designers, practitioners, and other end users work together to design initial prototypes and test them; and (3) appropriation studies that seek to capture long-term appropriation and the use of developed IT artefacts and any consequent changes in social practice (see again Walldius et al. 2005; Wulf et al. 2011; also Kuutti and Bannon 2014). In tandem with studies of the introduction and appropriation of designed artefacts, insights from the studies are also captured and used to inform potential redesign within relevant domains of practice. Here, the particular focus is upon the transformative impact of the functional elements and what options might exist for further development of the IT artefact. Individual design case studies are documented and analyzed in highly context-specific circumstances. Here we are interested in how the documentation and analysis might offer new opportunities for reflexivity, comparison, concept building and the transfer of insights.

Values and the e-Portfolio concept

The identification of cross-cutting issues in collections of design case studies so as to ‘compare and aggregate insights’ has been a long-standing concern at the University of Siegen (Wulf et al. 2011, 2015). Below we lay out the basic principles of the e-portfolio system. They reflect values derived from scientific discourse within HCI and related disciplines:

Self-reflexivity

Documentation of design and comparable processes enables reflexivity, which plays an important role in design (Schön 1983, 1987). It has also been discussed in relation to value sensitive design (e.g. Pommeranz et al. 2011; Borning and Muller 2012). Comparative analysis offers the possibility of cultivating self-reflexivity, thus stimulating a researcher’s creativity (e.g. Randall et al. 2018).

Sustainability

Large sets of data are often collected over the course of long-term projects. Such datasets mostly remain buried in digital folders once a project ends, which limits the potential usefulness of such material. Reuse can render much of what was previously invisible, visible.

Knowledge transfer

Comparison does not just foster the reuse of data, but also generates new knowledge such as the identification of similarities, differences and the rationale for former design decisions.

Encouraging practical research

Design research in different contexts can incorporate design concerns, technology-oriented outcomes, studies of appropriation, and so on. Rich descriptions of how design work is founded upon studies of social practice can help researchers to uncover both deeper and more diverse cross-cutting insights.

Accessibility for a broader audience

Academic publications are not the best medium by which to communicate findings and ideas to non-academic practitioners and citizens (e.g. Dachtera et al. 2014). However, well-documented studies offer other ways to transfer abstract theoretical concepts and research insights to these communities. In particular, there is an opportunity to provide more concrete examples to support common understanding, for instance through the use of images, videos and prototypes. This can make the benefits of design cases available to other groups outside the academic community.

It is also important to consider the values of the different parties who might make use of the e-Portfolio platform. It requires us to understand, analyse and negotiate the meanings that are otherwise opaque, in order to maximize the benefits of documentation. The stakeholders here are not just researchers and designers, but also practitioners, end-users, members of associations or NGOs, policy-makers, even ordinary citizens. Some of the values worthy of consideration here are: privacy, transparency, reputation, fairness, accountability, trustworthiness, provenance, and durability. Friedman et al. (2013) list a whole set of further possibilities, such as welfare, ownership, freedom, usability, autonomy, consent, identity, and sustainability. They also point to how design may result in both harms and benefits for different stakeholders and offer some initial ideas about how harms may be offset and benefits identified, including consideration of those who might be directly or indirectly impacted and those who have more or less knowledge and authority to deal with possible harms. This highlights a need for e-Portfolio to also consider these wider values and possible outcomes and to develop mechanisms whereby these different kinds of values can be handled appropriately and supported. e-Portfolio, by its nature, makes visible potential tensions and challenges between the values associated with the adoption of particular processes and tools over the course of various studies. This makes the value-laden character of tool and process choices available for further documentation and reflection so that future choices can be informed by their outcomes. Of course, at a meta-level, the creation of a specific e-Portfolio is itself open to examination in terms of the stakeholders involved in its creation and an examination of what values pertain to the act of assembling such a resource. The technical design of an e-Portfolio has to carefully address the visibility, accessibility and confidentially of data. Investigating the interests and tensions relating to different stakeholders facilitates sharing in a dynamic manner by helping us to understand how to share different kinds of material with different kinds of audiences.

Structure and design

The core of an e-Portfolio consists of layers, which are described in detail below. There are also connections that need to be built between the layers. In general, the idea is to provide functions that enable documentation of the evolving analytic and creative work that leads to the design of an artefact. The basic layer supports cooperative teamwork and the documentation of empirical data (audio recordings, interview transcripts, field notes, analytic categories, etc.) and technical design processes (e.g. scribbles, prototypes, and materials that underlie design decisions). By drawing upon information assembled in the basic layer, researchers can use the next layer to undertake comparative analysis across a range of case studies. The individual design cases may encourage researchers to evolve new perspectives and new ways of thinking that will help them to pursue processes of analytic comparison. Researchers can build new concepts that arise from understanding how IT artefacts are used in similar contexts. For example, by comparing the contextual details of two projects about aging, Wulf et al. (2015) were able to develop the concept of grown and constructed autonomy, which was used to characterize different aspects of care and caregiving in relation to support of the elderly. Insights such as this inhabit the next layer, providing for further conceptual reflection across other similar projects. It should be noted, however, that there is still work to be done regarding how to define project similarities so that they can be readily identified.

Challenges

As the e-Portfolio idea evolved, we noted significant challenges to its implementation:

Sensitivity of qualitative data

Researchers develop a very close relationship with their end users. A feature of this is that users do not just talk about matters relating to IT design, but also about their lives, their problems, their private circumstances, and anything that touches upon the research. Thus, their data can be highly sensitive. Conventionally this challenge is tackled through anonymization of the data. However, there are other issues that can arise. (A) The data may be so specific that even when anonymized it would still enable identification; (B) Anonymization is an overhead beyond transcription and, for raw data such as audio and video files, anonymization is much harder to accomplish; (C) Anonymization can sometimes undermine the legibility of the data, making it less valuable. So, while anonymization may work as a general policy, its effectiveness will vary from case to case. Another solution might be to extract high-level information from the raw data, but with the attendant risk that the richness of raw data is compromised.

Resistance to data-sharing

There are good reasons for researchers to share repositories of case studies, as noted above. However, this level of transparency may face resistance from some parts of the research community. It provoked controversy even within our own research group. To illustrate this point, in Table 1 we report verbatim some of the views expressed. As part of the development of the e-Portfolio design we have started to document both people’s existing data sharing practices, their attitudes to how data should be shared, and how the shared data can be reused by different groups (e.g. students, industry partners and citizens etc.). This is being undertaken systematically through a series of interviews and shows the specific problems associated with data sharing in the context of qualitative enquiries (Mosconi et al. 2019).

Table 1 Views upon data-sharing
Full size table

The divide between academia and practitioners

Practical research projects usually include IT companies and application partners (cf. Wulf et al. 2015). However, differences in design practice, organizational goals, or value systems among these parties can pose major challenges for conducting design research within what are typically short-term funding schemes (cf. Dachtera et al. 2014).

Transferring design-relevant insights to the “outside”

As noted above, academic papers with a specific focus are not the best medium for sharing all the design-relevant insights a research project can generate. A major issue is how to transfer such insights to practitioners that have not been involved in the research itself. We are still trying to identify effective mechanisms for sharing design-related knowledge with these ‘external’ parties through the documented materials. We are currently exploring whether specifically created explanatory notes, metaphors, or multimedia materials may help to fill this gap. In the development of UsersAward, Walldius et al. (2005, 2015) also presented materials in alternative formats as an aid to communication.

Incentive for data contributing

Moreover, as a result, design researchers who share material in the e-portfolio may have trouble getting their work acknowledged by the international research community, undermining the e-Portfolio’s basic goals. There remains considerable confusion about whether such shared material is ‘published’, consists of ‘pre-print’ material, etc. and this has ramifications when other publication outlets are sought. So, as the e-Portfolio idea develops into a distinct route whereby researchers may share their research, consideration will need to be given as to how to ensure that it may serve as a recognised and accredited form of output for researchers. We might add, therefore, the additional values of legitimation and reputation. In particular, there will be a need to explore how the relationship between e-Portfolios and conventional routes to publication should develop and to what extent e-Portfolios might need to be subject to external review. In many instances, researchers may want to have e-Portfolio outputs given the same status as published works, but this cannot happen without the wider community agreeing upon a mechanism. Once an understanding of the relative merit of an e-Portfolio contribution is in place, it will be easier to see how to assess individual contributions and the extent to which they might count when looking at matters such as hiring and promotion.

Negotiating and managing sharing between different stakeholders

A further challenge to be noted relates to the potentially divergent wider values that different stakeholders may express in relation to what kinds of materials are shared and the mechanisms used for sharing. Here, too, there is a need for further research to identify how e-Portfolios can best represent and manage these different concerns.

Conclusion

The rich documentation and knowledge produced by design cases studies in the context of grounded design have great potential, through comparative and reflective analysis, to enable the diffusion and production of further knowledge. At the same time, the domain of practice-based research actively demands comparative study and the communication of interdisciplinary knowledge. This work has therefore pursued the question of how to facilitate the transfer of single design insights beyond an individual level so that cross-cutting insights can be fostered, thereby offering further benefits to research. The solution we have pursued is the construction of a collection of design cases based on a bottom-up concept called an e-Portfolio. This portfolio can document cases, compare and identify their similarities, and enable sharing across different kinds of community, thus helping researchers to gain design insights from multiple perspectives and to transfer them to a more general level. This, ultimately, will serve to enrich interdisciplinary knowledge and foster lively discourse across the academic community.

The e-Portfolio concept seeks to expand researcher design insights, to promote the voice of practitioners and to build a bridge between academia and practice. Learning the different values in play and how they change according to different stakeholder contexts will help to drive VSD scientific discourses in the HCI domain and related disciplines.

The challenges noted above already indicate a number of areas where future work will need to be focused to turn the e-Portfolio idea into a fully-functioning and effective platform. Beyond this, and in line with the grounded design and design-based case study approaches we have discussed above, we will need to evaluate the e-Portfolio design by examining how it is used in practice, its effectiveness, the extent to which it is appropriated and, of course, the extent to which it itself becomes a driver of social change. We are already working on establishing a working e-Portfolio prototype that builds a collection out of materials available within our own group and makes them available in a variety of forms. This itself will be used to undertake early evaluation of its viability. This can then be used as the basis of an ongoing cycle of adaptation and refinement to meet user requirements, prior to rolling it out to a wider community and engaging in a similar process of study, evaluation and re-design.