Introduction: the need for data ethical case deliberation

The increasing implementation of ‘big data’ in management processes, research, and other areas challenges our traditional concept of how ethics should be applied and requires a specific approach addressing the possible unintended consequences that might arise from decision-making in data practices (Zwitter 2014). Raising concerns regarding big data-driven policy, Danah Boyd and Kate Crawford emphasize how the notion of ‘big data’ is changing what we consider to be true and factual (2012). They argue that the epistemological impact of data practices calls for a critical inquiry. Rob Kitchin has raised similar points, emphasizing the necessity for ethical reflection. In his words: “Here, there is recognition that research is not a neutral, objective activity that produces a view from nowhere, and that there is an inherent politics pervading the datasets analysed, the research conducted, and the interpretations made” (2014, 9). In addition to these general remarks on big data and ethics, a growing number of publications address specific cases wherein ethical issues arise from data sets, algorithms and the context of applied data practices, such as in financing (Pasquale 2015), in teacher evaluation and in recruitment (O’Neil 2016), or in social welfare (Eubanks 2018). While it has become increasingly clear that technological implementations come with challenges, at the same time it has been more difficult to see precisely what these specific ethical challenges are and how they might be practically addressed.

There are also objections to the lawfulness of government data projects in an international context.Footnote 1 These developments have given rise to concerns regarding the privacy of citizens but also question the possibly biased nature of such systems. The many issues related to data projects cannot be handled sufficiently by strictly applying privacy law and data management regulations. While privacy and the protection of personal information are well regulated by law, other uses of data, such as the application of algorithms or the inclusion of data subjects in the development of data projects are not necessarily regulated (Galloway 2017). Many aspects can be decided by the discretion of the administrators. And while some data practices are perfectly legal, they might not be morally accepted (Eubanks 2018). Especially in the area of both local and state government, laws do not cover all the issues related to a data project. Here, considering local values and how they are appreciated in each community might make regulation more relevant, or at least be an essential addition to it (Hendriks 2014). Municipalities differ not only in local issues and priorities, but also in the values that are represented in the policies brought forward by different councils (Meijer et al. 2019). How to deal practically with ethical challenges has been an open task.

Although ethical guidance is frequently sought, few, if any, dialogical frameworks aiming to increase ethical awareness amongst municipal employees exist. Furthermore, most of the existing frameworks are very abstract, emphasizing general values without considering the specific context of a data project and without facilitating practical application of data ethics. The aim of this paper is to respond to these shortcomings by asking: How can we make ethics practically applicable to government data projects? In this paper we provide a detailed description of the development of the Data Ethics Decision Aid (DEDA) framework for the Dutch governmental context, as an answer to our research question. Most importantly, this paper drills down more deeply into the details of how this framework has been developed, and evaluates its effectiveness. By doing so the DEDA framework can be seen as an applicable value-sensitive design framework for the context of governmental data decision-processes.

To illustrate how the framework was developed we will start by briefly reviewing positions on data and ethics. The specific context of the Netherlands will be sketched to provide context for this study and to indicate why the Netherlands is particularly interesting when looking at government data practices. The second part of this paper will describe how interviews at the municipality of Utrecht have led to the development of an initial prototype, which has then been adapted and improved in an iterative process of two focus rounds with civil servants. The final framework was tested with a user survey of 137 respondents to measure its effectiveness, while qualitative research into the effectiveness has also been conducted. In the third part of this paper, the framework itself will be described. Finally, we will discuss limitations and highlight gaps in the literature. We will claim that a framework like DEDA—and the findings that come out of using this process—will lead to further debate that enriches the scholarly discussions on data ethics.

Literature review

In this section, both the literature and theory that have influenced our work process for creating a data ethical case deliberation framework will be reviewed. First, we will theoretically situate our research within ethics-by-design debates. Secondly, the specific context of data projects in the Dutch government will be introduced.

The field of computer and information ethics has developed extensively over the last decade. Data ethics has been built on the foundation of computer and information ethics in order to maximize the ethical value of data science to society, people and environment (Floridi and Taddeo 2016). Floridi defines data ethics as:

[…] the branch of ethics that studies and evaluates moral problems related to data (including generation, recording, curation, processing, dissemination, sharing and use), algorithms (including artificial intelligence, artificial agents, machine learning and robots) and corresponding practices (including responsible innovation, programming, hacking and professional codes), in order to formulate and support morally good solutions (e.g. right conducts or right values). (2016, 3)

Floridi continues by stating three axes of data ethics, namely (1) the ethics of data, (2) the ethics of algorithms, and (3) the ethics of practises (Floridi and Taddeo 2016). All three axes have been considered in the development of the DEDA framework by looking into concrete practices in the context of municipal projects, examining how algorithms are used and understood, and walking through data life cycles to understand how data is used, collected, stored and shared. Nevertheless, in our research we have placed particular emphasis on the third area of data ethics, that is, the ethics of practices.

Value-sensitive design approach and ethical pluralism

The importance of including context to the data (e.g. Boyd and Crawford 2012), the critique of the objective nature of data (e.g. Crawford et al. 2014) and the notion that ‘neutral data’ is an oxymoron (Bowker 2013) have provoked debates about the issue of how context can be reflexively worked into data projects. In order to include context to the data and broaden the understanding of the “cooked” nature of data (Kitchin and Lauriault 2014) practical frameworks to structure dialogues are essential. This helps to deepen the insight of the personal interpretations of values and the necessity to reflect on one’s specific context. Therefore our research approach has been supported by two theoretical pillars, namely the value-sensitive design approach, and ethical pluralism.

The value-sensitive design approach can be described as an umbrella term for different scholarly debates most prominently held regarding values at play (Flanagan et al. 2005, 2008) and disclosive computer ethics (Brey 2000, 2010). ‘Ethics by Design’ understands ethical research not as a binary decision before the start of a research project, but as a continuous iterative process (d’Aquin et al. 2018). Based on the assumption that values are inherently built into technology, value-sensitive design approaches ask how to shape this process explicitly (Simon 2016). Therefore, the broader goal of value-sensitive design works on the expansion of the values that we use to judge the quality of technology (Friedman 1999). It can be understood as a: “theoretically grounded approach to the design of technology that accounts for human values in a principled and comprehensive manner throughout the design process” (Friedman et al. 2008). Value-sensitive design is concerned with questions regarding the wellbeing of participants and citizens, and focuses on human dignity, ideas of justice and human rights. The specific values that play a role include trust, accountability, freedom from bias, access, autonomy, privacy and consent. Most importantly, value-sensitive design aims to connect those who design systems and interfaces with other stakeholders (Simon 2016).

Instead of championing any particular ethical school, such as Kantianism, virtue ethics, utilitarianism, etc., we looked for ways that illustrate how outcomes differ depending on personal context and orientation and ethical reasoning (See for example in Ess 2006). Its underlying ethos might therefore be styled ‘ethical pluralism’ as defined by Ess: “ethical pluralism argues on a general level that the often striking differences between cultural beliefs, norms, and practices—e.g. regarding the nature of the individual and privacy—can often be harmonized by discerning how these differences may reflect distinct interpretations, understandings, and/or applications of shared norms and beliefs” (Ess 2020, 8).

Both value-sensitive design and ethical pluralism are important theoretical pillars from which this work draws its ethos and its inspiration.

Data and public management

Data practices are gaining relevance not only in the field of machine learning and AI, but also in public governance, as they are increasingly implemented. Local governments are transitioning to data driven public management, which potentially has a significant ethical impact on society and individual citizens (Mandinach and Gummer 2013). Such ethical challenges are, for example, the potential manipulation of citizens, wrong biases, and the lack of informed consent, to name just a few. Algorithms for predicting social welfare fraud, machine learning-driven algorithms for detecting deviant behaviour using surveillance camera footage, and data analysis to channel waste management and predict how subsidence affects real estate are just some examples of how data is used for prediction within the context of public management. With the rise of such data science, novel and complex ethical challenges emerge (e.g. Floridi and Taddeo 2016; Richards and King 2014; Zwitter 2014). Not only can data ethical guidelines be seen as an effective way to communicate the possible risks associated with big data to the public, but they can also help to increase the literacy of those working with data. There is an evident need for such ethical guidelines since legal policy often lags behind technological progress, leaving an expansive grey area where technology creates opportunities for new practices that have yet to be challenged by law and regulation. Moreover, the law does not regulate everything, leaving space for human actions to be guided by values rather than explicit laws. Not every unethical use of data is illegal, but can nevertheless be defined as undesirable in view of certain values or context (see also Van Schie et al. 2017). To give an example, while it is certainly legal to use certain types of surveillance measurements to gain insight into what employees do, it might not be ethical to do so.

Government organisations have access to data that ranges from basic citizen registers, land registries and census data, to data about public health, education, employment, criminal records, permits, and economic activity, etc. It cannot be emphasized enough that bureaucracies have always relied on data and data processing for carrying out their administrative tasks (e.g. Jacoby 1984; Agar 2003). Still, the amount of data, and the insights that are now possible to gain because of it constitute a very different paradigm. Critically questioning and examining (moral) dilemmas related to data projects and corresponding data practices constitutes data ethics. It allows one to deliberate on data projects in specific contexts and solve problematic aspects early in the project as well as anticipate the social impact. Given the complexity and entanglement of issues and stakeholders, making an ethical deliberation regarding the question of whether or not a project should be done is not an easy task.

Data ethics in the Dutch municipal context

Tasks and responsibility of municipalities in the Netherlands have been expanded as an effect of so-called decentralisation that took place in 2014–2015. While it was supposed to increase government efficiency through the management of a number of social responsibilities locally, this policy increased the influence of local governments over their residents. The national government delegated most social welfare tasks—such as youth guidance healthcare services for chronically ill and elderly residents, and unemployed persons assistance—to municipalities.Footnote 2 This caused municipalities to speed up their efforts to innovate public management processes in order to comply with the increased workload (Vermeulen 2015, 139; Maarse and Jeurissen 2016, 224).

Furthermore, it challenged and burdened the administrative organisation and IT-systems of municipalities as these had to facilitate a significantly higher number of tasks and management processes (Maarse and Jeurissen 2016).

Under these circumstances, data practices often promise to allocate resources from public funds to those who are eligible at the right time, predict where intervention by social workers might prevent additional costs, distribute scarce resources for public safety and order, and maintain municipal infrastructures cost-effectively. Efforts to innovate and digitize public management often entail “smart city projects” or the development of “data-driven management practices” (Timmer et al. 2015; Wesseling et al. 2018). The development of these initiatives is very asymmetrical, with only a few cities taking the lead in exploring possibilities for data-driven public management and actually developing working solutions for the application of data practices (ibid).

Local initiatives in the Netherlands have been, for the most part, exploratory and experimental, and insights and results have not been shared amongst municipal officials nationally, which raises questions of transparency and effectiveness. Reports from government advisory councils, and critical coverage in the media and from non-governmental organisations have recently criticized data practices on a national level. Even tax authorities have been subject to a governmental investigation, because a team of data scientists experimentally investigated taxpayers’ records without documenting their access to the data. It remains unclear whether private consultants participating in the data project were able to retrieve data for different purposes.Footnote 3 The City of Amsterdam committed a privacy breach by analysing Facebook data of teenagers loitering in public spaces.Footnote 4 Widespread criticism and parliamentary inquiries have been sparked by the use of the so-called Systeem Risicoindicatie (SyRi), an analysis process that uses a wide range of data sources to detect social benefit fraud.Footnote 5 In February 2020, this government-developed and government-used algorithm was rendered illegal by a Dutch court.Footnote 6

As a result, local governments have become increasingly aware of the need to handle data in a responsible and ethical way (van Noort 2015):Footnote 7 The City of Amsterdam introduced an ethical data manifesto,Footnote 8 a public register for municipal algorithms,Footnote 9 and guidelines for procuring algorithms.Footnote 10 The City of Zaanstad conducts ‘ethical reviews’ for its data projects with predictive qualities,Footnote 11 and the City of Utrecht made ethical reviews mandatory in 2018.Footnote 12 The Dutch Association of Municipalities (VNG) included data ethics into their data-driven management programme, and provided workshops using our data ethical case deliberation framework. Data ethics workshops were held during the annual conference of municipal Chief Information Officers of the four leading cities in the Netherlands, and at the G4 CIO Conference in 2016 and 2018. A data ethics workshop was also part of the annual congress of the Association of Dutch Municipalities in 2017 and 2019. Naturally, the European General Data Protection Regulation (GDPR) increased the urgency of governments (and other organizations) to review their data practices. In that context data protection impact assessments (DPIA) are useful to verify the GDPR compliance of a data project. However, a DPIA does not inquire into values carried or transformed by a data project. AI ethics guidelines or definitions of values to adhere to, appear to be rather abstract and difficult to apply in design processes. An applicable framework is therefore needed.

Method

To develop ethical guidelines as a practical tool, we followed an empirically driven research approach characterized by its openness towards the collected data (interviews, focus groups, surveys). Our field-work and general approach was partly inspired by ethnographic approaches to follow actors (e.g. Latour and Woolgar 1979; Latour 2005) and to shape societal impact through participatory action research (PAR) (McIntyre 2007; Baum et al. 2006). As a research method, PAR is distinctive in three particular ways. First, it is research that catalyses action. Action is achieved through reflective cycles: data is collected; it is then decided which action will follow; the action that is taken is then further researched and implemented before the cycle is repeated. Second, PAR is not blind to power dynamics: power is shared between the researcher and the researched. Third, this means that research participants are involved in selecting the research topic, the data collection and analysis, and decide what should happen next with research findings (ibid.). The purpose of such an approach is to build a framework that meets the needs of the practitioners and can directly be implemented into their everyday practice and organisational processes. Two needs have been identified: making participants in data projects aware of possible ethical pitfalls, and making the ‘political’ aspects—or values—carried by a data project explicit. To achieve this, we started with defining a criterion of how ethical awareness might possibly be measured. The most relevant criterion for the evaluation of our framework was:

Awareness of ethical issues in data projects: to which degree is the framework helpful in increasing awareness and knowledge of ethical issues in data projects amongst those who work with data?

The first round of expert interviews led to the development of the initial framework, which then was used in three rounds of focus groups, allowing for adaptation and further development of the framework. The first two focus groups were with the employees of a municipality, the last one with researchers and practitioners from Dutch academia.

Expert interviews

To better understand what civil servants needed from a data ethics framework, the aim of the interviews was to first understand where ethical issues occur, and what kind of awareness of ethical issues civil servants have. Eight expert interviews were held in December 2015 and January 2016 with key experts of the municipality of Utrecht. The selection criteria included their professional expertise in the field of data-driven management and their work with or for municipalities in the Netherlands. A pseudo-anonymised list of the participants is listed below. The interviews were loosely structured semi-structured and took place at their institutions. All of them were about 50 min long, and included a short introduction of the project. The interview guidelines consisted of ten questions: the opening question aimed to clarify the role of the interviewee within the institution, their work experience and their professional background. The second question block aimed to clarify their daily practice within these institutions and their self-understanding within the context of their work for the municipality. The third block explores ethical challenges, difficulties with other problems in this daily practice and the expectations of the employees toward an ethics tool. Subsequently, the findings of this first round of interviews have led to a rudimentary initial prototype that can be found in Fig. 1: Version 0.1.

Pseudonym

Description

P1

Data analyst whose company sells datasets to municipalities

P2

Data analyst Municipality

P3

Privacy Officer Municipality

P4

Security Officer Municipality

P5

Head of Quartermaster data-driven Management

P6

Information Process Manager Municipality

P7

Geo Data warehouse officer

P8

Open Data Coordinator

Fig. 1
figure 1

Overview of the various versions of DEDA

Full size image

Focus groups

To evaluate the initial prototype regarding the evaluation criteria, the same group of civil servants was asked for their feedback in order to retrieve information concerning the purpose of data ethics processes from project managers, analysts, data protection officers and policy advisors. The focus group was held in the institution itself and was one hour long. The input was implemented and the meeting was repeated thereafter. The second version became more modular, allowing users to consider various value-sensitive aspects of a data project (e.g. the issue of informed consent, bias or privacy). It also had a list of exit strategies for adapting the design of the project or discarding it. The same group of people was present during the second focus group. The dialogical aspect that it lacked was further developed and the framework was adapted for the next round.

The third version of the prototype was a table-poster inviting a collective discussion of the project. Feedback addressed the need to make easier to apply and to reduce the number of questions. For some, it still felt like a prototype: “Yes, it can be very useful but it needs improvements”. It was highlighted that the tool should be used as a group instead of as a solo activity: “Too much work needs to be done individually”. Feedback has been used to improve the process conceptually as well as to adapt the design. Figure 1 displays the process with the various versions that have been developed. The main changes consisted of developing a layout that stimulates a dialogical process, and combining this with the possibility to document deliberation either on a separate paper, or directly on the poster.

The framework has been frequently used, both for training participants in data ethics awareness and for revisiting data projects. The feedback from these use cases has led to further optimization of the process and the framework. During testing we noticed how important the role of the moderator was. The moderator can provide explanation when participants are not aware of why specific questions are part of the process, or when someone needs more background information. In addition, we developed a manual that introduces the questions and their motivation, and even mentions examples where they might be helpful to advance the understanding of the participants. Figure 2 shows the latest version of the DEDA framework.

Fig. 2
figure 2

The DEDA framework, version 2.0, 2020

Full size image

In November 2016, the first stable framework (DEDA 1.0) was presented to a group of ten researchers who are experienced in ethical issues surrounding data driven practices. They were asked to work with the framework and reflect on its usability and shortcomings. Generally, it was well received and did not lead to any more changes to the framework. A list of attendees to this focus group can be found below.

Pseudonym

Description

F1

A Dutch Prof Digital Humanities

F2

A Dutch Ethicist Big Data Rathenau Institute

F3

A Dutch Policy Advisor for the Humanities

F4

A Dutch Member of the Ethical Assessment Committee UiL OTS

F5

A Dutch Data manager institution for Open Society

F6

A Dutch PhD Student Copernicus Institute of Sustainable Development

F7

A Dutch Researcher Institute Network Cultures

F8

A Chinese PhD student Chinese Media Performance

F9

A Dutch Professor Media and Communication

F10

A German PhD Student Information Systems and Social Media

While this group could work perfectly well with “Step 3” in the DEDA framework, civil servants could not. Step 3 consisted of a summary of different ethical perspectives that could be applied to a data project, e.g. utilitarianism and virtue ethics. Initially, this step was supposed to make participants aware of the ambiguity of evaluating data projects from different angles, and to develop a theoretically informed decision. However, in practice this step was too complex and at times confusing to the participants, since generally speaking, civil servants do not have a background in philosophy or ethical theory. It has been replaced in version 2.0 by a review of the data project and the decisions formulated during the process in light of the explicit values of the participating organisation and its societal context. See Fig. 2 for version 2.0 of the DEDA framework, which is the version currently in use.

Evaluation of the effectiveness of the DEDA framework

After the development phase, the data ethical case deliberation framework was launched and several workshops using the framework were conducted with multiple organisations over the course of 2017 and 2018. In order to facilitate the dialogue, a researcher from Utrecht Data School guided all of the workshops discussed in this paper. The workshops are 3–4 h long and have the following structure: after an introductory presentation about data projects and ethics, the participants sit in groups of four to eight people with the DEDA framework on a table in the middle. Participants then follow the structure of the framework and discuss all of its questions, formulate answers to the questions and document them. They then present their findings to the rest of the participants and the facilitators, and invite the group to ask questions. The facilitators encourage participants to ask critical questions, and while doing so, reflect on values and ethical theories. The workshop ends with the participants deciding whether or not to continue with the data projects discussed, and under what conditions.

So far, Dutch municipalities comprise the majority of the organisations that have used the framework; however, workshops were also carried out within other government organisations such as ministries or the Directorate-General for Public Works and Water Management, educational organisations and several companies.Footnote 13

To analyse the effectiveness of the framework, a survey was created. In this section, we will discuss the survey and its responses. The survey was created on the basis of Bloom’s taxonomy, which introduces six different categories of knowledge (Bloom 1956). The first five categories: remembering, understanding, applying, analysing, and evaluating, were used to create the five questions of the survey. The questions were:

  1. 1.

    I know the benefits and pitfalls of data projects.

  2. 2.

    I can describe in my own words why data ethics has societal relevance.

  3. 3.

    I can formulate my gut feeling on the ethical pitfalls of data projects.

  4. 4.

    I am able to justify and account for decisions I take regarding data processing.

  5. 5.

    I have an informed opinion on data ethics and can advise others on data projects and their ethical implications, also when I am not directly involved in the project.

The participants answered the questions by selecting one of the options on a five-point Likert scale:

  • I totally agree

  • I agree

  • Neutral

  • I disagree

  • I totally disagree

In total, we collected 136 surveys during 12 separate workshops that took place in 2018 and early 2019. The data is entirely self-reported. The participants responded to the set of five questions before the workshop, and filled out the same five questions afterwards. The difference (increase or decrease) in knowledge was measured and used to gain insight into the effectiveness of the framework in conveying an understanding of data ethics. Reliability of the survey data was measured using Cronbach’s alpha and resulted in a final position of 0.787, which means the reliability of the survey data is acceptable (leaning towards good).

The user survey showed that using the DEDA framework increases participants’ knowledge concerning data ethics in all five categories of knowledge measured. Users with a low self-reported level of knowledge experienced the biggest increase in knowledge on data ethics. ‘Remembering’ and ‘evaluating’ were the most increased categories of knowledge, which correspond to Bloom’s first and fifth levels of knowledge, and relate to the first and fifth questions of the survey (1956).

Further research has been conducted to explore the effectiveness and added value of the DEDA framework to the development of responsible data practices in the Dutch governmental context. In 2019, Van Wijk compared three case studies in which the DEDA framework was used, and conducted eight in-depth interviews and document analysis to explore the effects the DEDA framework had on implementing ethics into data practices (Van Wijk 2019). Three specific case studies have been explored: one at a small municipality, one at a Dutch ministry, and the last one at a health organization. The study found that the DEDA framework stimulates reflection on complex issues in a structured way and that it makes data ethics concrete and applicable to specific data projects. Van Wijk also found some limitations of the DEDA framework and suggested improvements, like adding a question about the evaluation of data projects, and positioning it stronger as an ex ante framework. These suggestions have been taken into account by the developers of the DEDA framework.

The DEDA framework

In the upcoming section, the data ethical case deliberation framework will be introduced. During the interviews with the employees of the municipality, it became clear that a guiding framework was needed for the employees to increase awareness of the process of data projects and involved ethical issues, unify that process and learn how to show the public that data is not used arbitrarily: “People expect developments, we cannot sit still!” (P3). A decision-process is needed because: “We have a lot of information to deal with. Only a little bit is useful for our purpose. We have to ask the right questions to get the right answers. But with so much information it’s hard...” (P4). Respondents pointed out that a dialogical tool would help them. While the very first prototype excessively resembled a generic list of values and rules about the responsible treatment of data, it provided little actual guidance for revisiting a specific data project. Figure 1 provides an overview of the various versions that have been developed. Figure 2 shows the current version, version 2.0, which will be elaborated on in the upcoming section.

Introducing the Data Ethics Decision Aid (DEDA)

The design and development process resulted in the DEDA (Fig. 2). It is a framework for a dialogical reflection on data projects in the governmental context. It consists of a poster (see Fig. 2) displaying a list of questions concerning data-related issues and general considerations. The process that participants walk through consists of three major steps. Step one is clarifying who will document all decisions made, step two serves to position the project within a specific context through specific questions around various ethical aspects. The last step serves to compare institutional values with one’s own and reflect on the practice.

The DEDA framework in its current form (Fig. 2) is a size A0-poster. It contains 11 clusters and a total of 46 questions, and is designed to be placed on a table amidst a group of participants. We observed during the second focus group that a mere list of values alone provides little guidance for employees. Nevertheless, values can be an interesting starting point for reflection. On the table poster we therefore start by stating the values and then clarify with additional questions. The full list of questions can be found in the appendix. We created some additional tools that can be used to support data ethical case deliberation, such as a manual that gives a clear explanation on how to use the DEDA framework and provides background information on certain concepts used. The third focus group has pointed toward the danger of a “bottleneck of control”. It was feared that ethics could serve as mere impact assessment and not actually provide ground for reflection (F1). Therefore, the framework is explicitly not a checklist, since each question requires an extensive response, as opposed to a yes/no answer. Furthermore, it encourages group deliberation and is expressly not meant to be used by a person working on their own.

The first set of questions (the blue part at the top of the poster) is related to data, data quality, the source of the data, the use of algorithms or models, data visualization, data management, and re-use of datasets. The lifecycle of datasets are considered, because ethical questions change with them. This became clear in the second focus group. Special sections for the collection of data, the use of data and the reuse of data are marked. Questions regarding the algorithm, for example, might be more relevant in the first phase of a project, whereas questions regarding the visualization process might be more relevant later in the project. The second set of questions (in green) addresses issues such as responsibilities, communication strategies, social impact, bias, privacy and informed consent. Step three invites the participants to explicitly benchmark their data project against the values of their organization. A brief description of different ethical concepts allows for the thought experiment of how a data project is perceived through different normative frameworks.

Choosing a spiral form to organize the clusters of questions on the poster stimulates dialogue and involvement in the process, as all participants are invited to read the questions aloud and thus actively engage in the deliberation, because not all questions are legible for all participants at the same time. The spiral design is therefore a deliberate design decision that attempts to break through hierarchies within teams between, for example, male and female participants, or bosses and employees. It is designed to create an open and friendly space in which deliberation is possible. This has proven to be helpful in encouraging every team member to speak up and raise their concerns.

The process of using the DEDA framework

The aim of the process is twofold: first, it raises awareness and conveys knowledge about (the complexity of) data ethics. This aim has been shown to be effective by the survey that was conducted amongst 136 users. Second, it encourages and facilitates a structured dialogue around ethical aspects of data projects. The framework is explicitly not a checklist, but centres around a joint dialogue and decision-making process. This is grounded in value-sensitive design theory.

The DEDA framework is designed in such a way that participants start with the context and understand the complexity of the issues involved. Importantly, however, focus is directed towards a solution or a decision-making process. The design encourages debate, but tries to guide people through the process without losing their attention or giving them the feeling that issues are irresolvable. Users learn to communicate with each other on ethical aspects and to foster a safe environment for voicing critical questions. Results of a session using the DEDA framework consist of clear action points for the participants. During the deliberation, one participant is asked to take notes and write down all decisions and considerations made. This can be used as documentation regarding the ethical case deliberation and decision-making for the specific data project in question.

We found that most civil servants tend not to look clearly at informational material that is provided but want to be walked through the process. For that reason it became clear that the DEDA process requires the supervision of a facilitator. We see the danger of looking for loopholes and running through the process by simply postponing unsolved issues for later reflection. A facilitator and expert can critically engage with the debate and also serve as moderator by asking critical questions or summarizing debates. We have seen that processes with a facilitator bring the most fruitful outcomes and have therefore decided to make it obligatory to run through the DEDA framework with a facilitator first. After answering all the questions on the DEDA framework, and once it is clear what next steps need to be taken, a facilitator is no longer necessary. For later sessions the Utrecht Data School offers “train the trainer” workshops to introduce team members into the facilitation process and prepare them for this role.

A team of people working together on a data project within a municipality or another governmental organization can use the DEDA framework in the early stages of the project, or during an evaluation. The project must be in a development or redevelopment phase, because only then can the ethical pitfalls that have become apparent by using the framework be mitigated by changing (aspects of) the project. At this moment in time, the framework is being used by over 30 government organizations in The Netherlands. Some organizations have even made the DEDA framework an obligated step of the design process of sensitive data projects.

In practice, public sector professionals are swamped with policies and regulations, such as the Allgemeine Bestuurs Recht, and the General Data Protection Law.Footnote 14 The DEDA framework cannot be seen as just another impact assessment, or tool to keep in mind. Instead, it directly asks professionals which other guidelines and laws need to be reflected on (See appendix DEDA table poster Question 22: “Which laws and regulations apply to this project?”)

Discussion

Value-sensitive design has long pointed toward the need to reflect on how design choices influence ethical reflection. Values are not independent from their socio-cultural context and they are implicit in decision-making processes (Turiel 2002). In order to make values held by different stakeholders explicit, methodologies are needed to increase better understanding and trust in data processes (Dignum 2018). Therefore, reasoning and reflection on data processes should: “take into account societal values, moral and ethical considerations; weigh the respective priorities of values held by different stakeholders in various multicultural contexts; explain its reasoning; and guarantee transparency” (Dignum 2018, 1). We believe that the DEDA framework can be seen as such a methodology, and as far as we know it is the only approach that works explicitly in that direction. The DEDA framework and our approach, however, are not without limitations. These will be discussed in the next section.

Limitations of the DEDA framework

By asking questions and adding context, the DEDA framework responds to the need for context-sensitive methodologies. Instead of prescribing what needs to be done, it prompts debate and fosters reflection on personal gut feelings and data life cycles. The process of articulating personal concerns and being provided with additional information on concepts like algorithms and anonymization helps people to consciously choose their next steps. Similar to the ethical approaches of AoIR (Ess 2002; Markham and Buchanan 2012; Franzke et al. 2019), the DEDA framework tries to ask the right questions. Asking questions, however, can also be seen critically. The open rights group has published a white paper in which different ethics guidelines are discussed. In this paper, concerns regarding the nature of the questions in the DEDA framework have been raised.Footnote 15 Referring to the survey results, and our experience of working with the framework now for several years, we are convinced, however, that the open and broad approach of inquiring into a data project through different categories of questions is well-suited to kick-start an ethical deliberation. Its strength lies in introducing important concepts, and facilitating constructive debate, more so than attempting to deliver clear-cut answers or merely state a set of core values to adhere to.

There are certainly limitations to our attempt to evaluate DEDA’s effectiveness. With regard to the user survey that measured the difference in awareness and knowledge about data ethics before and after using the DEDA framework, it is important to consider that the answers were completely self-reported by participants, thus providing subjective interpretations of one’s own knowledge. Besides, what data literacy means varies according to context and ongoing technological developments (Jaeger et al. 2012). Our empirical experience with DEDA is strongly determined not only by our expertise as developers but also as moderators of DEDA workshops. Hence, we have not sufficient information about how others are working with DEDA and whether it yields similar results. Additionally, a strong focus on case deliberation approach could come with the danger of inconsistent policies because one team might solve a case differently from another team somewhere else. This raises concerns regarding the possibility and also the need for generalizable data practices and top–down regulations. We are positive, however, that the DEDA framework can be one of the first steps to increase awareness about issues in very concrete cases, which lead later on to more generalizable reflections. These then feed into best practices and top–down regulations. The fine-grained insight into specific practices allows for more nuanced discussions as opposed to abstract thinking about broad values.

We have learnt that the DEDA framework in itself is as good as the known ethical and technical challenges at a time and believe that it is important to constantly improve and further develop a dialogical ethics framework like DEDA. In our case it meant to include communication between municipality and citizens as an aspect to consider deliberately during the deliberation process. In the first versions of DEDA, we were unaware of the urgency to address this aspect in order to develop ethically responsible data practices. Recently, a dimension time became an urgent issue to address: self-learning algorithms might change in a way that they do not meet the criteria of the ethics assessment anymore, or a data model reflecting the political values of the municipal council requires changing after the next election. And this becomes even more urgent when the emphasis of the ethics discourse is changing. So far, fair, transparent or explainable algorithms and data projects have been at centre of desirable data practices, but recently an emphasis has been put on assessing their emancipatory quality (Kalluri 2020).

The plethora of other ethics manifestos and tools

There are many processes, tools, guidelines and manifestos communicating how to pay attention to values and social impact when applying data analysis and developing artificial intelligence applications.Footnote 16 Most of these are guidelines emphasize general values to consider when working with AI, and are very similar to the EU’s High Level Expert Group on AI recommendations. In general we can distinguish three different sets of ethics guidelines: (1) the manifestos and guidelines providing rather abstract general frameworks with little or no guidance for practical implementation, (2) checklists and canvasses, and (3) modular and process-oriented frameworks with a clear perspective on practical application. The DEDA framework falls into the last group. While DEDA revolves literally around a canvas, it is the elaborate structure of a workshop, the role of the moderator and the significant effort to develop a conversation and deliberation process considering design choices, responsibilities, and accountability, that enables clear perspective towards a value-sensitive design process. As such it differs from other tools in the field. Organisations have often already looked at different options among the available ethics guidelines before consulting the Utrecht Data School. Most frequently mentioned is Tada,Footnote 17 a process developed by the City of Amsterdam and the Economic Board Amsterdam in Cooperation with Waag Society. The Data Ethics Canvas of the Open Data Institutes (ODI), with its appealing and user-friendly design, is also often mentioned. The efforts of the EU Commission to formulate guidelines through its High Level Expert Group on AI and their seven key requirements for trustworthy AI also play a role in the field.Footnote 18 The Dutch research institute TNO developed a modular framework in form of a card game in 2014, and recently the Utrecht University of Applied Sciences developed a similar modular version, also as a set of cards.Footnote 19 We only mention these ‘card games’ because they have also been part of our brainstorming process, and we appreciate the modularity of this approach. However, we have not encountered organisations making use of it, and consider it more appropriate for data ethics awareness training. Ethics scholars Merel Noorman and Linnet Taylor (2020) recently analysed Tada’s blind spots. While they laud the emphasis, Tada puts on paying attention to core values when designing, they criticize Tada’s lack of guidance in making these applicable to individual cases’ distinct contexts. Here, the DEDA framework fundamentally differs from Tada and other manifestos or guidelines, including the EU’s guidelines for trustworthy AI. While those try to provide a broad, abstract set of rules, the DEDA framework starts with making the organisation’s values and practices explicit, and reflects thoroughly on the context in which the organisation operates and in which the data practices will be deployed. A comparative analysis of accountability mechanism for algorithms aptly concludes: “This means that instead of posing explicit and general norms for responsibility, DEDA provides a process of raising ethical questions. These questions are, of course, implicitly normative. Nevertheless, they appeal to the ethos that an organization has and wants to maintain instead of postulating norms.” (Buhmann et al. 2019). The brainstorming and deliberation process facilitated by the DEDA framework gives leeway for unintended consequences and unforeseen effects through pre-emptive design.

Comparison with the Data Ethics Canvas

Another tool worth mentioning is the Data Ethics Canvas by the Open Data Institute. It was one of the first tools available for reflecting on data practices. It consists of a poster, which also has seen some changes over time. The poster displays fourteen areas to consider when working with data (see Fig. 3). Many of those overlap with the set of questions addressed on the DEDA poster. While we find the Data Ethics Canvas very useful in providing a set of essential areas to consider when working with data, the DEDA framework goes further in its engagement of the participants. It always starts with the reflection process on values and practices within the organisation and its context. The workshop and the design of the poster is deliberately shaped to connect the different participants and their respective agendas, and facilitate a conversation across different departments and levels within a hierarchy. The Open Data Institute clearly noted the need for a more thorough implementation of its canvas into the organisations of their users. They now provide a training course on the application of the data ethics canvas.

Fig. 3
figure 3

The Data Ethics Canvas by the Open Data Institute

Full size image

The DEDA framework sets itself apart from the ODI’s Data Ethics Canvas by separating data-related issues from general considerations and responsibilities to locate within the organisation developing the data project. This helps organisations decide on clear action points during the workshops, which can then be taken up by members of the organization who have the appropriate skill set (see DEDA in action, Fig. 4). It is also visible in the poster design which deliberately provides blank space on its canvas to attach action points, decisions, and requests for further information, or resources directly under the discussion points. The DEDA framework stands apart in its effort to enable dialogue and facilitate documentation of the deliberation process. Through documenting the deliberation and design processes of data or AI projects, DEDA provides an essential feature for constituting accountability. A critical audience (e.g. journalists, council members or citizens) could request information about the design decisions for inspection or verification. This practice could inform the traditional checks and balances with the very much needed transparency to safeguard democratic values and practices in a digital society.

Fig. 4
figure 4

DEDA process; different municipal employees (data analysts, data protection officer, policy advisor, domain expert, alderwoman, and project manager) evaluate a data project

Full size image

Ethics is an important variable at play in decision-making processes, but so are efficiency/costs considerations, compliance, and transparency. Nonetheless, we believe that making implicit values explicit and demonstrating how these affect the outcome of a data project is one of the big benefits of the DEDA framework.

Dutch governmental context vs. international and corporate context

A useful framework in one context might not have the same strength in other contexts. However, we see how the DEDA framework can be used in various contexts outside of the Netherlands and not be limited to public administration. An EU Horizon 2020 project report recommends using the DEDA framework for developing codes of conduct for digital citizenship (Colman et al. 2018). The U.S. American Federal Data Strategy explicitly mentions the DEDA framework as a useful resource in a recent draft version of their data ethics framework.Footnote 20 In an international academic context, the Association of Internet Researchers indicates that the DEDA framework has informed a recent edition of their code of conduct (Franzke et al. 2019). The DEDA framework has not yet been sufficiently tested in the corporate world, but in a comparative overview of “Business Process Tools Mapped to IFC Technological Code of Conduct”, technology providers and decision makers are identified as the key target audience for DEDA (Myers and Nejkov 2020, 6). Our own experience with commercial partners indicated a significant challenge for using the DEDA framework. While most municipal and government employees share a commitment to the explicit values of their organisation and can agree on their civic role, companies often have difficulties to identify shared values.

Conclusion

This paper has outlined the iterative development process of the DEDA framework and its evaluation. We have demonstrated an applicable ethics tool for data practices. Its relevance manifests in translating concepts of ethics by design, value-sensitive design and ethical pluralism into an applicable process. We have shown that the DEDA framework increases data ethical awareness. Contrary to other data and AI ethics guidelines, DEDA does not propose a strict set of rules and core values. It begins by making explicit the values that govern the organization wherein data practices are being developed. It provides an applicable process of adhering to values, adapting design, documenting design decisions, and implementing organizational changes to constitute ethical data practices and accountability. As such, the DEDA framework appears to be very compatible with the existing guidelines, which manifest core values to consider, but fall short when it comes to providing means of application.

Our examples have shown that DEDA is already effectively used in a number of municipalities and educational organisations in the Netherlands. As the DEDA framework is freely available online, it is difficult to track how it is used and by whom. So far, we have not implemented it structurally in other areas outside public management, nor outside of the Netherlands. However, positive feedback at various international conferences and workshopsFootnote 21 indicates that the framework might be useful in other countries and contexts. Shedding light on the complex decision-making processes within institutions, and determining if the DEDA framework would fit into these contexts and what kind of adaptations would be necessary, is research that needs to be done in the future. Engaging with the DEDA framework might also be useful for ethics scholars, allowing them to observe empirically how ethics are addressed when developing data practices and implementing algorithms. These insights might inform the conceptualization of an ethics for a digital society.